-- Logs begin at Tue 2022-07-26 15:24:06 UTC, end at Tue 2022-07-26 16:58:03 UTC. -- Jul 26 16:12:34.493036 dv-cinder-851042-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 16:12:34.502294 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 16:12:34 2022] *** Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: nodename: dv-cinder-851042-1-iscsi Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: machine: x86_64 Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: clock source: unix Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: pcre jit disabled Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: detected number of CPU cores: 4 Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: current working directory: / Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: detected binary path: /usr/bin/uwsgi-core Jul 26 16:12:34.504272 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: your processes number limit is 47680 Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: your memory page size is 4096 bytes Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: detected max file descriptor number: 2048 Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: lock engine: pthread robust mutexes Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: thunder lock: enabled Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 16:12:34.505119 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 16:12:34.544291 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: Python main interpreter initialized at 0x5642e6a259c0 Jul 26 16:12:34.544291 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: python threads support enabled Jul 26 16:12:34.544291 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: your server socket listen backlog is limited to 100 connections Jul 26 16:12:34.544291 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: your mercy for graceful operations on workers is 90 seconds Jul 26 16:12:34.544656 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 16:12:34.544732 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: *** Operational MODE: preforking *** Jul 26 16:12:34.545007 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: *** uWSGI is running in multiple interpreter mode *** Jul 26 16:12:34.545007 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: spawned uWSGI master process (pid: 80356) Jul 26 16:12:34.545210 dv-cinder-851042-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 16:12:34.546468 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 1 (pid: 80368, cores: 1) Jul 26 16:12:34.546468 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 2 (pid: 80369, cores: 1) Jul 26 16:12:34.546468 dv-cinder-851042-1-iscsi devstack@keystone.service[80356]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 16:12:40.094237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:12:40.094237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: from cryptography.utils import int_from_bytes Jul 26 16:12:40.095120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:12:40.095120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: from cryptography.utils import int_from_bytes Jul 26 16:12:40.099166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:12:40.099166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 26 16:12:40.099626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:12:40.099626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 26 16:12:40.144776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 16:12:40.144965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 16:12:40.145158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] command line args: [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 16:12:40.145425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 16:12:40.145602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ================================================================================ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 16:12:40.145853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] admin_token = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.146098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_dir = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.146288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.146471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] config_source = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.146651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] debug = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.146850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.147105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 16:12:40.147171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 16:12:40.147227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.147366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] command line args: [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 16:12:40.147446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.147576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 16:12:40.147669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.147736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ================================================================================ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 16:12:40.147844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.147988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_dir = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_config_append = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_source = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_dir = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] debug = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.148915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.149925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_config_append = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_dir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.150830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.151972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.152927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.144/identity {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] publish_errors = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.153902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] standard_threads = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.154982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.144/identity {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] publish_errors = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_journal = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_json = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_stderr = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] use_syslog = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.155894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.155970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.156041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.156116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] standard_threads = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.156267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.156353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.156534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.156695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.156695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.156872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.156951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_journal = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.157022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.157090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_json = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.157197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_stderr = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.157276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.157375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_syslog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.157454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.157608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:12:40.157687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.157864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.157944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.158917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.159918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.160872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.161987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.162938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.163971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.164871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.165996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.166925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.167962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.168133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.168312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.168486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.168711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.168861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.169859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.170895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.171958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.172981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.173963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.174921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.175938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.176994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.177940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.178893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.179016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.179141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.179259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.179375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.179497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.180933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.181897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.182950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.external = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.183899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.password = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] auth.token = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.184902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.external = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.185965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.password = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.token = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.186935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.187992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.188877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.189947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.190988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.191924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.192987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.193945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.194889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.195005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.195121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.195923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.196912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.197994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.198957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.199904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.200878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.201993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.202921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.203991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.204969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.205923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.206958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.207887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.208885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.209825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.210881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.211942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.212901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.213927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.214993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.215983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.216948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.217774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.218967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.219882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.220995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.221929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.222917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.223974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.224941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.225900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.226917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.227985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.228958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.229921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.230994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.231934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.232926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.233952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.234894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:12:40.235252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 16:12:40.260598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.260829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.261061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.262100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.262337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.262565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.269455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.270892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.298752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.298981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.299999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.300236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.353591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.354091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.403686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.403916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.405209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.405445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.468155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.468413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.468707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.468852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.468925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.469968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.492964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.493930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.494063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.494142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.494276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.505579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.505706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.518884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.518994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.519973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.520067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.520184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.520267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.520428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80369) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:12:40.537150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.537362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:12:40.537645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.537728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.537857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:12:40.537953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:12:40.538140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.538220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.538383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:12:40.538463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.538649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.538729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.539439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.539525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.539697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.539953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.549142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.549419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.549691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.549812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.549973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.550097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.550369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.550649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.557488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.558169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.562433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.562712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.562994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.563238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.563321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.563520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.563621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.563783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.563863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.564974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.565144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.565223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.565412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.565684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.565957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.588173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.589323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.589693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.589971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.590826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.591104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.594600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.594878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.595159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.595733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.596014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.596292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.602330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.602605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.603424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.603741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.606733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.607018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.607309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.607588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.607860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.607954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.608988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.609269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.609536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.609811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.625290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.625567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.626266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.626543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.629514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.630458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.632716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.633023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.633625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.633919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.637417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.638303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.640738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.641491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.643974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.644275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.644588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.644671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.644878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.644959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.645156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.645240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.645450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.645533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.645795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.646080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.661560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.661839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.662055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.662149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.662335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.662416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.662598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.662860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.669812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.670422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.671344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.671618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.671953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.672228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.674481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.674761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.675332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.675908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.676207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.687928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.688207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.688479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.688564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.688759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.688840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.689856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.690128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.705322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.705688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.708291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.708665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.711293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.711638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.713772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.714098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.715295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.715594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.715675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.715876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.715957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.716152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.722404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.722628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.724009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.724222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.727489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.727770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.727852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.728056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.728137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.728329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.728429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.728625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.736470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.736685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.736780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.736972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.737053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.737262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.744457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.744658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80369) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:12:40.744742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.744934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.745857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.746955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.747845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.748023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.748103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.748279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80369) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:12:40.781538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642e6a259c0 pid: 80368 (default app) Jul 26 16:12:40.781647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642e6a259c0 pid: 80369 (default app) Jul 26 16:12:40.784797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72471c-19e8-4efc-b6bf-ad9980236f6b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:40.785087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72471c-19e8-4efc-b6bf-ad9980236f6b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:40.785363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72471c-19e8-4efc-b6bf-ad9980236f6b None None] PATH_INFO: `/v3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:40.790260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1/1] 10.222.0.144 () {50 vars in 796 bytes} [Tue Jul 26 16:12:40 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:12:49.305219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:49.305780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:49.305780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:49.310295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1/2] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:49 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:12:49.315677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:49.316075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:49.316237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:49.385996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: INFO dbcounter [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] Registered counter for database keystone Jul 26 16:12:49.407162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG oslo_db.sqlalchemy.engines [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80368) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 16:12:49.413464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writer thread running {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 16:12:49.588342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80368) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 16:12:49.588711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80368) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 16:12:49.642019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:49.643602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:49.776791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 2/3] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:49 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 16:12:49.919625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:49.988723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: INFO dbcounter [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] Registered counter for database keystone Jul 26 16:12:50.090294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG oslo_db.sqlalchemy.engines [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 16:12:50.090294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writer thread running {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 16:12:50.180853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881087-5401-4277-ba65-d83b37baf6cf None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:50.186181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:50.186830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:50.187111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:50.187381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:50.195969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:50.247884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING keystone.common.rbac_enforcer.enforcer [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 16:12:50.248218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da268c-260b-40dd-8d35-db1c28ae8baf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:50.258325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 2/4] 10.222.0.144 () {58 vars in 1329 bytes} [Tue Jul 26 16:12:49 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:12:50.264249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:50.278569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef098a9-32f7-4846-ae64-d502d98b2d0a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:50.283047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:50.284143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:50.284477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:50.284759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] PATH_INFO: `/v3/projects/bf7f222516294286bf361e9d63865e13` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:50.286275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] RBAC: Authorizing `identity:get_project(project_id=bf7f222516294286bf361e9d63865e13)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:50.337826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 16:12:50.338401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50e3e-29b7-4827-9b64-73e932ef7083 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:50.342741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 3/5] 10.222.0.144 () {56 vars in 1212 bytes} [Tue Jul 26 16:12:50 2022] GET /identity/v3/projects/bf7f222516294286bf361e9d63865e13 => generated 345 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:52.315123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957afdbe-baf4-4e69-80d3-791717266e10 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:52.315634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957afdbe-baf4-4e69-80d3-791717266e10 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:52.315736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957afdbe-baf4-4e69-80d3-791717266e10 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:52.320495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 3/6] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:52 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:12:52.325884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:52.326347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:52.326441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:52.346159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:52.347662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:52.355448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 4/7] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:52 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 16:12:52.377480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957afdbe-baf4-4e69-80d3-791717266e10 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:52.390098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957afdbe-baf4-4e69-80d3-791717266e10 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:52.394355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:52.394953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:52.395237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:52.395512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:52.403095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:52.404940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf033181-6330-488d-b3aa-fd4c13fb179e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:52.413569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 4/8] 10.222.0.144 () {58 vars in 1329 bytes} [Tue Jul 26 16:12:52 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:12:52.418736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:52.427566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514281c-65d0-425c-bff8-ed3dab5b6397 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:52.432473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:52.433095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:52.433377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:52.433653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:52.434999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:52.437096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12af58-7d9d-4a35-b48a-1cb5aacc1a7d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:52.438614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 5/9] 10.222.0.144 () {56 vars in 1203 bytes} [Tue Jul 26 16:12:52 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:12:54.515803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8a8b0-ab1e-4dc5-99ef-3edecdf128dd None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.516279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8a8b0-ab1e-4dc5-99ef-3edecdf128dd None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.516376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8a8b0-ab1e-4dc5-99ef-3edecdf128dd None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.517745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 5/10] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:54.522948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.523370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.523622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.527811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10814a4-4d9f-47a5-a593-cd916f5b80cf None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.528102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10814a4-4d9f-47a5-a593-cd916f5b80cf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.528401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10814a4-4d9f-47a5-a593-cd916f5b80cf None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.529648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 6/11] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:54 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:12:54.534591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.534881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.535162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.542716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:54.543904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:54.577309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG passlib.handlers.bcrypt [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80369) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 16:12:54.577744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG passlib.handlers.bcrypt [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80369) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 16:12:54.621775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 6/12] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:54.623068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:54.624698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:54.643567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:54.666537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:54.725260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 7/13] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:12:54.746709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f6dc4-568e-4aec-9499-b42e5a41b32c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:54.747992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:54.780353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:54.780991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.781272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.781551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.846564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d0d13-e0ae-484e-8d86-0f30668ee87e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:54.848709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:54.848834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:54.850264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:54.873574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:54.885670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:54.886765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:54.887052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:54.887337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:54.900510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:12:54.909143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:12:54.910100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:12:54.910984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 8/14] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:12:54.916133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:54.949707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 7/15] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:12:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:12:54.955136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:55.014225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdaab4-a081-410a-9e78-114f7c4bae4c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.047538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.048151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.048456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.048733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.049444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:12:55.049748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.049836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894707-61f3-4070-a3a2-15b2660075a2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.051177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.063548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 9/16] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:12:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:55.069079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:55.071931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:55.082786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.083416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.083694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.083960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.094026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.094715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:55.095512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:12:55.113924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:12:55.114885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:12:55.115681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:12:55.115681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:12:55.115681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:12:55.115681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:12:55.115681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 8/17] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:12:55.120845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:55.169857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca41e-702d-4484-9534-84905749ea68 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.206392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.207058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.207340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.207611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.209280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.210912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.212703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 10/18] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:12:55 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:55.218533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:55.320731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:55.326296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb08-0c5f-4453-8f20-c056a42dafe7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.330261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402e79c-234f-4516-9645-3d180c66dd62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.359275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.359918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.360209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.360514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.361647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.363105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.364115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.364732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 11/19] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:12:55 2022] GET /identity/v3/domains/default => generated 200 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:55.364835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.365039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.365311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.366005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:12:55.366816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.368191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1982327-ea64-4a66-b69f-0080e8b7f4e2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.370864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:55.386127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 9/20] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:12:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:55.474961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad78e2-7366-447a-9eed-6b5d81611375 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:55.511544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:55.512176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:55.512487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:55.512767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:55.513579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.domain.id=default, user.id=d88063924d80461f80d0ee08a35cdcfc) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:12:55.513891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:55.515370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d3c6-8e0f-45b6-a9f6-0f8a3161cdb5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:55.524488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 12/21] 10.222.0.144 () {56 vars in 1327 bytes} [Tue Jul 26 16:12:55 2022] GET /identity/v3/role_assignments?user.id=d88063924d80461f80d0ee08a35cdcfc&scope.domain.id=default&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 234 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:12:57.797791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f480-c7c8-455c-9933-c9643be14d83 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:57.798647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f480-c7c8-455c-9933-c9643be14d83 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:57.799067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f480-c7c8-455c-9933-c9643be14d83 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:57.800584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 10/22] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:57.805808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:57.806606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:57.807075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:57.833419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:57.835586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:57.863472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:57.921957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 13/23] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:57.958421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f480-c7c8-455c-9933-c9643be14d83 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:58.002885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:58.089178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f480-c7c8-455c-9933-c9643be14d83 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:58.141626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:58.142266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.142547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.142818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.153488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:58.154943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5fbceed9-380a-400a-9182-780b21b9857d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:12:58.169920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:12:58.170723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:12:58.171497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:12:58.171497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:12:58.171497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:12:58.171497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:12:58.171497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 11/24] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:12:58.176913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:58.242289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db482508-88cf-4458-b822-dd254ec7d0d6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.242600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db482508-88cf-4458-b822-dd254ec7d0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.242848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db482508-88cf-4458-b822-dd254ec7d0d6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.244485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 12/25] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:58.253549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.253838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.254113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.297416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:58.298868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:58.304733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96253d02-eea5-40cd-8e60-bb1cb673563e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:58.350023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:58.351430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.351860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.352279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.353156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:12:58.353769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:58.355334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:58.400137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 14/26] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:12:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:12:58.406517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:58.440004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:58.454680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 13/27] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:12:58.533265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:58.564653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:58.702496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d35009-338b-4629-a8dc-7e71e213243c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:58.713003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a318e6-1f4c-4445-8eab-5ee152709af9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:58.739551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:58.740273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.740730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.741201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.742174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:58.743837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef40b45-4724-4443-b531-ed3d28b3a3dd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:58.754364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:58.754988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.755278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.755534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.756219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:58.758308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdb525-819d-4932-a3fb-de6a759b1196 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:58.941156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 14/28] 10.222.0.144 () {60 vars in 1136 bytes} [Tue Jul 26 16:12:58 2022] POST /identity/v3/roles => generated 226 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:12:58.943338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59b41-8729-4f37-8982-8e35a46ffcda None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.943867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59b41-8729-4f37-8982-8e35a46ffcda None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.944925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59b41-8729-4f37-8982-8e35a46ffcda None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.946906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 15/29] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:58.948608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802710e-15a4-41e7-b551-bf3d262f2c15 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.949133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802710e-15a4-41e7-b551-bf3d262f2c15 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.949597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802710e-15a4-41e7-b551-bf3d262f2c15 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.951696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 16/30] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:58.953277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3a6f-31b5-4bb1-ba8c-e357faf67807 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.953818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3a6f-31b5-4bb1-ba8c-e357faf67807 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.954313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3a6f-31b5-4bb1-ba8c-e357faf67807 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.956535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 17/31] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:58.957930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340a52-7450-4b90-806b-3ed5210ce027 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:58.958418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340a52-7450-4b90-806b-3ed5210ce027 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:58.958911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340a52-7450-4b90-806b-3ed5210ce027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:58.979261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-97340a52-7450-4b90-806b-3ed5210ce027 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:58.981287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-97340a52-7450-4b90-806b-3ed5210ce027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:59.059064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 15/32] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:12:58 2022] POST /identity/v3/projects => generated 302 bytes in 653 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:12:59.061803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b993-fece-42f2-9208-260ed56f4d4c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.062341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b993-fece-42f2-9208-260ed56f4d4c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.062813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9b993-fece-42f2-9208-260ed56f4d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.085308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3eb9b993-fece-42f2-9208-260ed56f4d4c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:59.087273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3eb9b993-fece-42f2-9208-260ed56f4d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:59.137793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 18/33] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:59.140662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28110a-961f-4a6a-bc5c-861e5f7b23a3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.141423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28110a-961f-4a6a-bc5c-861e5f7b23a3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.141934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28110a-961f-4a6a-bc5c-861e5f7b23a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.163523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.173963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7c28110a-961f-4a6a-bc5c-861e5f7b23a3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:59.175203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7c28110a-961f-4a6a-bc5c-861e5f7b23a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:59.218982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.231967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 16/34] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:59.234168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e3bcd-8b15-43f0-bc1d-f4013b365b0b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.234459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e3bcd-8b15-43f0-bc1d-f4013b365b0b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.234737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e3bcd-8b15-43f0-bc1d-f4013b365b0b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.249715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 17/35] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:12:59 2022] GET /identity => generated 271 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:12:59.249715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e3bcd-8b15-43f0-bc1d-f4013b365b0b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.296335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 19/36] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:59.297968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.298258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.298535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.318266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:12:59.320468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:12:59.382478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e3bcd-8b15-43f0-bc1d-f4013b365b0b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.393886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 20/37] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:12:59.394733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.421640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.422266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.422546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.422819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.423534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:59.425075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:59.466053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.499501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71e9dc-c116-4b41-a3b3-1e3a4638af85 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.508518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 18/38] 10.222.0.144 () {60 vars in 1136 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/roles => generated 220 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:12:59.509327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.537083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.537755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.538577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.538856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.553322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:12:59.562903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:12:59.564036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 21/39] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:12:59.565528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.580156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.629013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfedcae-2a10-47ee-8072-33e635fc7285 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.669371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.670008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.670297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.670577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.671304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:59.672761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:59.681967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a626c-9f27-4929-9818-78261cf76b5f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.708358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.717537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.718179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.718550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.718833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.719999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:59.721612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:59.723191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 22/40] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:12:59 2022] GET /identity/v3/domains/default => generated 200 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:12:59.723948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.781389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 19/41] 10.222.0.144 () {60 vars in 1136 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/roles => generated 224 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:12:59.782379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.836085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98d90-3fc0-4fbc-96b2-39bd52d1f4a0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.874066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.874709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.874993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.875272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.876000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:12:59.876309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:59.877751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:59.892108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 23/42] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:12:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:12:59.898625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:12:59.913685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc732f-4a0c-447a-bc0f-66742d86ee97 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:12:59.932513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:12:59.953140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:12:59.953774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:12:59.954060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:12:59.954397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:12:59.955089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:12:59.956643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:12:59.966299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:00.013735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490c5f8-a94f-447c-ba20-1f325e957ba8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:00.046896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:00.047480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:00.047766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:00.048049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:00.049257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:00.050716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:00.052301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 24/43] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:12:59 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:00.058147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:00.086325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 20/44] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:12:59 2022] POST /identity/v3/projects => generated 313 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:00.162247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e38d1-c070-42f1-b9d5-676462411dfa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:00.187731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:00.198416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:00.199077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:00.199364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:00.199643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:00.200912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:00.202411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd4a8-1b2a-488a-8873-dbd2c1b11713 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:00.204058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 25/45] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:00 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:00.209962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:00.313180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fae9fd-5a61-4e49-9419-13a03e4ec8f4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:00.345979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:00.346662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:00.346952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:00.347236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] PATH_INFO: `/v3/domains/default/users/d88063924d80461f80d0ee08a35cdcfc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:00.349250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d88063924d80461f80d0ee08a35cdcfc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:00.351247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62786ab8-e494-4ca0-903a-2e1755c0071b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:00.422044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 21/46] 10.222.0.144 () {58 vars in 1366 bytes} [Tue Jul 26 16:13:00 2022] PUT /identity/v3/domains/default/users/d88063924d80461f80d0ee08a35cdcfc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:02.091701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff5df-2c75-4921-98dc-6979afc76945 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.092212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff5df-2c75-4921-98dc-6979afc76945 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.092293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff5df-2c75-4921-98dc-6979afc76945 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.093638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 26/47] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:02.098982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.099415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.099672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.117945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:02.119130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:02.141760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:02.185397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 22/48] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:02.207836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ff5df-2c75-4921-98dc-6979afc76945 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:02.314047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ff5df-2c75-4921-98dc-6979afc76945 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:02.350015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:02.350629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.350908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.351184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.352335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:02.353812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2ad04-0e9c-4f80-a7b9-8935acc5f00a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:02.355443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 27/49] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:02 2022] GET /identity/v3/domains/default => generated 200 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:02.361235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:02.406175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6cb2-40fd-40ae-a978-13894bff2154 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.406464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6cb2-40fd-40ae-a978-13894bff2154 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.406742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6cb2-40fd-40ae-a978-13894bff2154 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.408071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 28/50] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:02.412694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.412985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.413260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.431392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:02.432610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:02.458991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b16405-b3e0-4112-8039-af181fb4fd6c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:02.488958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:02.495504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:02.496205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.498214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.498214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.498214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:02.498978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:02.500625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 29/51] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:02.524495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:02.581671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 23/52] 10.222.0.144 () {60 vars in 1145 bytes} [Tue Jul 26 16:13:02 2022] POST /identity/v3/projects => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:02.629345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dbfa6-9531-4417-95a8-052154efc3ac None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:02.666040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:02.666699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.666984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.667264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.679311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:02.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:02.689348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:02.690157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 30/53] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:02.692644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:02.710552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:02.797494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03cac-69f9-4115-bccc-0caa92aad139 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:02.831534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:02.832150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.832458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.832735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.833465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:02.833826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:02.835233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:02.843408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 24/54] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:02.849223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:02.899680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:02.955296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421135-f50d-41a4-acea-ea5ff3b9bb10 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:02.996710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:02.997662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:02.997952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:02.998230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:02.999463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:03.001368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:03.008066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 31/55] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:02 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:03.009299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:03.083693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:03.107994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063c74a-7bc2-4fe1-8e92-50a86202eb2a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:03.141014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:03.141648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:03.141931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:03.142205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:03.143339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:03.144813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbf011-7e54-4dbc-b7fc-ff11c4ae9b9c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:03.146385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 25/56] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:03 2022] GET /identity/v3/domains/default => generated 200 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:03.152228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:03.250030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c817c-58a8-474f-a890-bb5aebdbef38 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:03.268204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:03.284985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:03.285643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:03.285923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:03.286195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:03.286978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.domain.id=default, user.id=d88063924d80461f80d0ee08a35cdcfc) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:03.287284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:03.289367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a8f54-10bc-4aed-9270-4494e20f9e1e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:03.298710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 32/57] 10.222.0.144 () {56 vars in 1327 bytes} [Tue Jul 26 16:13:03 2022] GET /identity/v3/role_assignments?user.id=d88063924d80461f80d0ee08a35cdcfc&scope.domain.id=default&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 532 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:04.587422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b8bb-beec-433c-9106-970436fe1c13 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:04.587965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b8bb-beec-433c-9106-970436fe1c13 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:04.587965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b8bb-beec-433c-9106-970436fe1c13 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:04.589379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 26/58] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:04.594278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:04.594645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:04.594833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:04.615636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:04.617422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:04.680763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 33/59] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:04.705244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b8bb-beec-433c-9106-970436fe1c13 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:04.805938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b8bb-beec-433c-9106-970436fe1c13 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:04.837669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:04.838297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:04.838582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:04.838860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:04.840005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:04.841480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a19cc-45ad-427c-92e6-83b036cf478b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:04.843050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 27/60] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:04 2022] GET /identity/v3/domains/default => generated 200 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:04.848578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:04.943410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a5a1-55ac-425b-a8e3-2c655cd17c75 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:04.956528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:04.977300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:04.977913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:04.978198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:04.978473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:04.979166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:04.980692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205285-482f-486d-a9cb-7cc2d149c8ee None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:05.056105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 34/61] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:04 2022] POST /identity/v3/users => generated 276 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:08.159916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466eb5e-ad1e-489e-896e-ba5f04b101ba None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.160954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466eb5e-ad1e-489e-896e-ba5f04b101ba None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.161473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466eb5e-ad1e-489e-896e-ba5f04b101ba None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.163536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 28/62] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:08.169439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.170276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.170746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.193051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:08.195191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:08.279914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404030d-8391-4abb-b331-aef681e45f2d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.280587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404030d-8391-4abb-b331-aef681e45f2d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.281059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404030d-8391-4abb-b331-aef681e45f2d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.283749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 29/63] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:08.288288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1f6c-c6dc-4ef1-a265-3cad24cf4297 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.288816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1f6c-c6dc-4ef1-a265-3cad24cf4297 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.289361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 35/64] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:08.289755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1f6c-c6dc-4ef1-a265-3cad24cf4297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.317631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3f9f1f6c-c6dc-4ef1-a265-3cad24cf4297 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:08.319163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3f9f1f6c-c6dc-4ef1-a265-3cad24cf4297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:08.325349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:08.341775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.422752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 30/65] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:08.441146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f122-ce9d-47e1-8fb5-d6cf787ddf96 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.441778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f122-ce9d-47e1-8fb5-d6cf787ddf96 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.442305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f122-ce9d-47e1-8fb5-d6cf787ddf96 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.444145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 31/66] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:08.449612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.450126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.450617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.469923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:08.471547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:08.484045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:08.489135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9f9d-4412-4e46-af5a-f4789216100b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:08.530825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:08.531711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.532560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.532982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.546986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:08.554983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:08.556000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:08.556945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:08.559538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 36/67] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:08.564172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.566051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 32/68] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:08.573391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.670540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:08.690945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff0398-1c55-4c04-9029-f0ba8233ee66 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:08.711956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116ffe-ed29-4152-918d-a1e2aa5a5c21 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:08.735908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:08.737244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.737864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.738491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.750665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:08.751288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.751571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.751850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.752734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:08.753045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:08.753553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:08.754431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886c3e0-3465-4805-a0e1-c669b37b3e22 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:08.759113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:08.760027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:08.760801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 37/69] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:08.761399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.768767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 33/70] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:08.768767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9e59-4d0c-4418-9560-bb7100d11d6a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.768767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9e59-4d0c-4418-9560-bb7100d11d6a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.768767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a9e59-4d0c-4418-9560-bb7100d11d6a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.768767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 34/71] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:08.769150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b115b-9efb-4204-8efd-9ec7fffa43ee None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.769386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b115b-9efb-4204-8efd-9ec7fffa43ee None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.769578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b115b-9efb-4204-8efd-9ec7fffa43ee None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.770777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 35/72] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:08.771598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b115b-9efb-4204-8efd-9ec7fffa43ee None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.782082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:08.861080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab62f-901b-4612-9aff-ea27fcbc7445 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:08.871603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b115b-9efb-4204-8efd-9ec7fffa43ee None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:08.895712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:08.896341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.896645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.896917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.900923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:08.905688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:08.906334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.906622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.906901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.907444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:08.912314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:08.915340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 38/73] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:08.916248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:08.923301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:08.924799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e3b16-e634-4937-865f-7033e700eefa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:08.926483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 36/74] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:08.927652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800785-3e92-48db-94e0-61cf170715af None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:08.927944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800785-3e92-48db-94e0-61cf170715af None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:08.928762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800785-3e92-48db-94e0-61cf170715af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:08.946977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-17800785-3e92-48db-94e0-61cf170715af None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:08.948106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-17800785-3e92-48db-94e0-61cf170715af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:08.997284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.014392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 37/75] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:09.015893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.016188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.016489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.017671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40175a35-ac9f-4c56-a777-8672f007c00a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.049709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:09.051100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.051409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:09.051717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.051998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.052271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.053017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:09.053888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.055380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.063607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 39/76] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:08 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.064400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.124594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 38/77] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:09.125744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.173602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.184396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757631-95fc-4c46-845a-bc2abcc672b5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.218617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.219243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.219696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.220038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.220878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:09.221186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.222572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.224116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc91610-81d5-4d03-86b8-ec07db450a90 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.230252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 40/78] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.231085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.233714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.258985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.259718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.260081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.260456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.329090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.330564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.331428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e3be-e0bb-448a-85d0-411e53b4d49d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.364018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.364690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.364976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.365249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.375675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:09.381049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:09.381903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 41/79] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:09.383066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.390853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.441510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 39/80] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:09.442372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.443123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.481332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bbe9-73e9-432c-a706-53a78ac987b3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.516144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.516826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.517121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.517399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.518576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.520038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.521681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 42/81] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.522983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.548129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a894-3065-4d19-8b59-2f4c081e620a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.581154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.581783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.582057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.582329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.583463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.585478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.591650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 40/82] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/domains/default => generated 200 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.591650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.609549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.619705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e97f31-7e5d-4509-846c-194475ed1d7b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.647676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.653655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.654295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.654582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.654868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.656038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.657534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.659141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 43/83] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.659906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.689091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820ca8-5816-47ce-b70f-e3b10f43f3a5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.721995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.722608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.722889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.723163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.723880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:09.724183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.726177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.734065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 41/84] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.734792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.764262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608980b-9e21-463f-9683-6a36db124aac None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.796930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.797561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.797841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.798113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.799253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.800722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.802357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 44/85] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:09.803123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:09.814390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.833366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3932-0f7a-4934-9846-8358ec2e4190 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.867219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.867826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.868108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.868397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.869181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:09.903884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d2e6-aa82-4cb0-a61d-959c468a3455 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:09.934000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.935691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:09.936300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:09.936936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:09.937222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:09.937494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:09.938175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:09.939571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.013099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.021016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 45/86] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:09 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:10.022712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.033289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 42/87] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:10.034489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.152030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a41702-7374-423c-a1f4-dc4b64000cca None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.159091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3a227-9fd8-49f2-9977-a36bca44ad1c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.185246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.195733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.196180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.196251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.204836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.204836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.205062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.205162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.205238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.205238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.205238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.205410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 43/88] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:10.205410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.258303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.285098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.285795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.323387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3053aa-7256-4357-ae56-46c0a9ab503f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.364507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.365226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.366663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.366663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.367093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:10.367404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.368875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.374557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 44/89] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=5a1e89be85c547438247b00c04c09046 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:10.375372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.419547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 46/90] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:10.421083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.451414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.499362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f44857-269f-4cab-b0f7-1d3c83e1ecf8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.514918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.539524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62befaad-f884-49b2-a291-763788391b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.540190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62befaad-f884-49b2-a291-763788391b47 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.540591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62befaad-f884-49b2-a291-763788391b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.541030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62befaad-f884-49b2-a291-763788391b47 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.542848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62befaad-f884-49b2-a291-763788391b47 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.544662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62befaad-f884-49b2-a291-763788391b47 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.547896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 45/91] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:10.548847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62befaad-f884-49b2-a291-763788391b47 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.550211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd02dd-8127-4428-8517-3aa6d070a58e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.589469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.590200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.590482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.590756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.661909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62befaad-f884-49b2-a291-763788391b47 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.667034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.668647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.706620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.729250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.729922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.730209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.730538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.731816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.733303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.734993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 46/92] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:10.735764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.805429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 47/93] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:10.806199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:10.826986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.934188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460902e-cdbd-467b-bd57-6d00b5e3920d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.944871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb27e9-a126-4b98-baae-fb903d11f1b2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:10.958835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:10.971266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:10.971886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:10.973421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:10.973421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:10.973421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:10.973823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:10.975238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:10.981118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 48/94] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:10.982008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:11.040476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:11.040476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:11.040476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:11.040476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:11.040838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=d88063924d80461f80d0ee08a35cdcfc) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:11.040838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:11.041949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:11.052802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 47/95] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/role_assignments?user.id=d88063924d80461f80d0ee08a35cdcfc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:11.100320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba650-bd52-42cf-aff6-07533a66b105 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:11.139646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752467e0-466a-4946-8888-0655447db82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:11.140316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752467e0-466a-4946-8888-0655447db82d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:11.140626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752467e0-466a-4946-8888-0655447db82d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:11.140900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752467e0-466a-4946-8888-0655447db82d None admin] PATH_INFO: `/v3/projects/8b50b2d802354944913b873c17f2b5bc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:11.142208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752467e0-466a-4946-8888-0655447db82d None admin] RBAC: Authorizing `identity:get_project(project_id=8b50b2d802354944913b873c17f2b5bc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:11.143734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752467e0-466a-4946-8888-0655447db82d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:11.145724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 49/96] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:10 2022] GET /identity/v3/projects/8b50b2d802354944913b873c17f2b5bc => generated 313 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:11.151934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:11.280765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477135ea-6b68-4060-9c9c-9a1fab06d0f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:11.295682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:11.319589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=8b50b2d802354944913b873c17f2b5bc, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:11.324047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8562b04-95f7-44e0-9b1b-dbe2fbeea995 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:11.330532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 48/97] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:11 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=8b50b2d802354944913b873c17f2b5bc&role.id=5a1e89be85c547438247b00c04c09046 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:12.815482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c96ed-75ba-4abd-9d76-9bdd0137be94 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:12.815978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c96ed-75ba-4abd-9d76-9bdd0137be94 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:12.815978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c96ed-75ba-4abd-9d76-9bdd0137be94 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:12.817473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 50/98] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:12.829625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ed334-301f-4ae3-814a-9f20b404958d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:12.830044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ed334-301f-4ae3-814a-9f20b404958d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:12.830235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ed334-301f-4ae3-814a-9f20b404958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:12.857315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-901ed334-301f-4ae3-814a-9f20b404958d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:12.858519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-901ed334-301f-4ae3-814a-9f20b404958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:12.941757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 49/99] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:12.972284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c96ed-75ba-4abd-9d76-9bdd0137be94 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:13.040758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:13.112203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c96ed-75ba-4abd-9d76-9bdd0137be94 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:13.136282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744a66-7cbd-4590-837f-edfaf705bbef None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.136613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744a66-7cbd-4590-837f-edfaf705bbef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.136883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d744a66-7cbd-4590-837f-edfaf705bbef None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.138968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 50/100] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:13.144667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.145429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.146111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.180418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:13.180418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.180418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.180418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.181196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:13.182962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:13.184459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 51/101] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:12 2022] GET /identity/v3/domains/default => generated 200 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:13.189935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:13.191636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:13.193388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:13.293131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 51/102] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:13.348167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:13.368157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ff03-57f5-4d24-9ab8-644f4dc6afef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:13.392418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:13.406071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:13.410044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.410044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.410044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.410044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:13.410044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37d8c7-1c9f-43bb-b239-28b105238c45 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:13.446846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:13.497304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015713f6-7858-47b2-995f-2a27ef3cff52 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:13.513018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 52/103] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:13:13 2022] POST /identity/v3/projects => generated 303 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:13:13.514748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00ee1-2e63-45fb-914d-a5ccc954cf16 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.515181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00ee1-2e63-45fb-914d-a5ccc954cf16 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.515643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00ee1-2e63-45fb-914d-a5ccc954cf16 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.517367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 53/104] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:13.526468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.527013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.527469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.562554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:13.564124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:13.586784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:13.587852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.588506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.589186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.626246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:13.639974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:13.640906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:13.641639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:13.643706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 52/105] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:13.650054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:13.688505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 54/106] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:13.712885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:13.882598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:13.892526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dd280-0193-4f1c-a6a1-155962b89e1e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:13.957712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:13.958344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.958628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.958900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.959692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:13.959958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:13.965413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ae32-3763-4198-98d5-12ddfc25b8c7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:13.965996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ca961-e971-4445-ab89-44e78a2f2827 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:13.969439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:13.979610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 53/107] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:13 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:13.980899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90413f-a753-41eb-9913-c6e24825704e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:13.981196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90413f-a753-41eb-9913-c6e24825704e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:13.996313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90413f-a753-41eb-9913-c6e24825704e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:13.996313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 54/108] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:13.996313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90413f-a753-41eb-9913-c6e24825704e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.011635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.012650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.013186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.013799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.026483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-286e11cb-7ec4-4b01-9e4d-53e0b8a38ecf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:14.032001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:14.033014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:14.033756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:14.036017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 55/109] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:14.038508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.039002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.043307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.058939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:14.060804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:14.162214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 56/110] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:14.163025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.166237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90413f-a753-41eb-9913-c6e24825704e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.229360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.230012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.230302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.230584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] PATH_INFO: `/v3/users/b36e39c714814374bc4c6b1cee0be089` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.257696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] RBAC: Authorizing `identity:get_user(user_id=b36e39c714814374bc4c6b1cee0be089)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.259170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.261119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 55/111] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:13 2022] GET /identity/v3/users/b36e39c714814374bc4c6b1cee0be089 => generated 290 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:14.261694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.292233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.334311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.343572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b126c-c379-4fa6-a8f0-132282b2336e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.383916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8cf55-f678-436e-8767-5c08aa0717d2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.390504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.391215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.391565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.391907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.392729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:14.393107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.395098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.406870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 57/112] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:14.407598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.425353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.426019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.426380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.426826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.441204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-34a92b26-1a06-49a0-b490-b8c3bd605aa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:14.448509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:14.449543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:14.450484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:14.452026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 56/113] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:14.452684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb092e-7edd-4863-b4ca-34e454c4b5c1 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.455430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb092e-7edd-4863-b4ca-34e454c4b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.455430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb092e-7edd-4863-b4ca-34e454c4b5c1 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.455430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 57/114] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:14.456073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb092e-7edd-4863-b4ca-34e454c4b5c1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.530156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56637b97-b18c-4703-b62e-7818e1cd58aa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.565587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.572936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.575719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.575719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.575719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.588359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb092e-7edd-4863-b4ca-34e454c4b5c1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.608845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.624621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.625263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.625537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.625811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.626963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.628438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.630036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 58/115] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:14.630793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.668511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.670233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd6f3d-8fc0-45fa-8bc9-6b1791248be9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.738339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1a55c-0a16-4914-8dc0-a0aca66e0d91 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.774284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.774898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.775264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.775544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.776716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:14.777030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.778761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561e9fd-bb20-420a-8f56-f8ff3e272fc5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.788224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 59/116] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:14.789772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.790065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.790331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.804412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 58/117] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:14.804412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0094bd-5026-4f44-bbd6-9c975c2f7457 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.804412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0094bd-5026-4f44-bbd6-9c975c2f7457 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.804412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0094bd-5026-4f44-bbd6-9c975c2f7457 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.804904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.808435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 59/118] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:14.808435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0094bd-5026-4f44-bbd6-9c975c2f7457 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.815498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:14.816969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:14.872919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:14.890346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 60/119] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:14.901618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:14.924899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0094bd-5026-4f44-bbd6-9c975c2f7457 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:14.964113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:14.964779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:14.965063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:14.965339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:14.966662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:14.968043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:14.974672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 60/120] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:14.974672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.018122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59dc18-ae18-45eb-beac-bbab9619d929 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.054924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.055665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.055953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.056236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.057464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.058922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc993e-92a0-4781-a6f1-96238c3a98f9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.065168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 61/121] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:15.065168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.065168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.065168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.070240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.081809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:15.085053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:15.085189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23d844-5ee3-4f7f-b462-560d43825118 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.110466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.122679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.127294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.127294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.127294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.127294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.127294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.134015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 61/122] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:14 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:15.134015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.154257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 62/123] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:15.162365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.251228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3624ef1-6df5-471c-9f00-458ff4dcec60 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.261155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6424-bbc5-4778-a7c2-8292b58ade01 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.286116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.286685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.286971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.287275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.297167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.297837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.298117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.298277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:15.298376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/7806a5c324534a3fa0c5c442f35fb401/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.300302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, user_id=7806a5c324534a3fa0c5c442f35fb401, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.301803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:15.303602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:15.304476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:15.305298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:15.307504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.309662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 62/124] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:15.310626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.362516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.386471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 63/125] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:15 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/7806a5c324534a3fa0c5c442f35fb401/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:15.396804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.431470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79711f7f-c69b-4170-a8e7-c344393d9388 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.488063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.488917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.489312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.489694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] PATH_INFO: `/v3/projects/8b50b2d802354944913b873c17f2b5bc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.493375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] RBAC: Authorizing `identity:get_project(project_id=8b50b2d802354944913b873c17f2b5bc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.495125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.497526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 63/126] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/projects/8b50b2d802354944913b873c17f2b5bc => generated 313 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:15.498563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.544142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df21a-30f4-48cd-8b83-f56eec2512a6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.590860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.591272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.591438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.591708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.592531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=b36e39c714814374bc4c6b1cee0be089) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:15.592833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.594296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.605786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 64/127] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/role_assignments?user.id=b36e39c714814374bc4c6b1cee0be089&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:15.606620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.619020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb23280-bb63-44f5-bb67-1c2e17b2bcac None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.650051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.681430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.682094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.682376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.682651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.711181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:15.717672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:15.726157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 64/128] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:15.726897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.743647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:15.826223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea4a-a3b3-4171-8bb0-bdd1bbf74b09 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.870697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.872129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.872716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.873177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.874232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:15.875115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.876723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:15.889000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 65/129] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:15.890223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:15.897532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6fa6c-e10a-44d8-aa80-7e82f354e706 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:15.945757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:15.946773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:15.947336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:15.947866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] PATH_INFO: `/v3/projects/8b50b2d802354944913b873c17f2b5bc/users/7806a5c324534a3fa0c5c442f35fb401/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:15.950255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] RBAC: Authorizing `identity:create_grant(project_id=8b50b2d802354944913b873c17f2b5bc, user_id=7806a5c324534a3fa0c5c442f35fb401, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:15.952057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8f9d-99da-48c5-9179-b65fb711ad7f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.010836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fff39-aee7-4061-844e-52b415f02013 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.032375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.047016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 65/130] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:15 2022] PUT /identity/v3/projects/8b50b2d802354944913b873c17f2b5bc/users/7806a5c324534a3fa0c5c442f35fb401/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:16.054954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3870042-9f84-4f8c-a99f-e0877127ec83 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.055356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3870042-9f84-4f8c-a99f-e0877127ec83 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.055554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.055888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3870042-9f84-4f8c-a99f-e0877127ec83 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.056534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.056997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.057419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.057739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 66/131] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:16 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:16.058477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:16.058594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3870042-9f84-4f8c-a99f-e0877127ec83 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.059029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.061172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17c7c3-407c-4abd-86a2-55e9816369da None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.074844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 66/132] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.076562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.077038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.077454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.100289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:16.102564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:16.160109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.196963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 67/133] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:16.199146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.302667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3870042-9f84-4f8c-a99f-e0877127ec83 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.352468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758cd7d-0f3f-4eaf-9102-6fe791841570 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.354783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.356551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.356981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.357396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.358775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.360672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.362934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 67/134] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:16 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.364227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.373692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.406886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.408304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.408717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.409352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.410930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.413296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.415087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 68/135] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:16 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.416224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.516673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9e48-2a83-40c0-a6a4-45fd6dff1cde None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.529020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.582469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d753-3983-4d05-9c06-046afd1b3581 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.612802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.613116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.613372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.613646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.614877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.617043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.618912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 68/136] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:16 2022] GET /identity/v3/domains/default => generated 200 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.619590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.637899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.638884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.639396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.639883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.641322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.643000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.649195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 69/137] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:16 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.652645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.760827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a231f-a831-4322-bb37-e42ca05a449f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.776437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.822181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.822681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.822973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.823253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.829166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.830652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.832025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f152b0-8a9d-46a4-889f-32ac35133197 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.832417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 69/138] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:16 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:16.833221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:16.869423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:16.870172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:16.870457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:16.870813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:16.871497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:16.873076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:16.913128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:16.970338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ff31b-a72b-4d37-901e-f392c256f8a5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:16.988108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 70/139] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:16 2022] POST /identity/v3/users => generated 284 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:16.993373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:17.060673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:17.061408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:17.061695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:17.061972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/7806a5c324534a3fa0c5c442f35fb401/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:17.067497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, user_id=7806a5c324534a3fa0c5c442f35fb401, role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:17.069219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e54c-b7ab-4536-a263-f8dc3b254315 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:17.155191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd4779-789c-45e4-ae47-c4b8a54ab4a9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:17.157836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:17.170626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 70/140] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:16 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/7806a5c324534a3fa0c5c442f35fb401/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:17.202313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:17.203017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:17.203351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:17.203633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/d88063924d80461f80d0ee08a35cdcfc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:17.205728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, user_id=d88063924d80461f80d0ee08a35cdcfc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:17.207251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08910a2-060e-49b5-9372-610c72fabe1f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:17.313648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 71/141] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:16 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/d88063924d80461f80d0ee08a35cdcfc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:17.742458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709fd0-6e3a-47ba-8cc7-44d499151923 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:17.742815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709fd0-6e3a-47ba-8cc7-44d499151923 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:17.743139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709fd0-6e3a-47ba-8cc7-44d499151923 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:17.748935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 71/142] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:17.750083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:17.750373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:17.750649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:17.773439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:17.774797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:17.856781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 72/143] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:17.906686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709fd0-6e3a-47ba-8cc7-44d499151923 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:18.022745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709fd0-6e3a-47ba-8cc7-44d499151923 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:18.025111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:18.066482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:18.067944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.068495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.068970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.085278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:18.090599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:18.091417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:18.092164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:18.094373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 72/144] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:18.100478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:18.220480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:18.228832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fa697-c0ce-493c-9cc6-9348c308fa51 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:18.269172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:18.270328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.270328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.270328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.271080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:18.271407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:18.279994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2a3d6-a6d6-4df8-abed-782e38a52907 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:18.285494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 73/145] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:18 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:18.291598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:18.403108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe91d1e-ebe8-426e-8238-6daefb9493c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:18.450148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:18.450798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.451078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.451352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.452562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:18.454014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:18.455912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 73/146] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:18 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:18.460199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb275-ecb3-4d7f-ab4f-c6549799b1a2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.460515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb275-ecb3-4d7f-ab4f-c6549799b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.460791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb275-ecb3-4d7f-ab4f-c6549799b1a2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.462131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 74/147] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:18.465420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:18.467196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.467477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.467751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.509613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:18.510837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:18.522177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:18.655008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176908bf-b698-4731-b2ae-11b0c1d77463 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:18.682770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 75/148] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:18.718528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:18.719161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:18.719439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:18.719711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:18.738124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:18.835761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:18.837263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:18.901564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:18.951425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:18.984058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bde1b0-1e5e-4653-adae-9bbb16da7054 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:19.072505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 74/149] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:18 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:19.074793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:19.075450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.075735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.075998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.077633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.109330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:19.114331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:19.115691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:19.116881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 76/150] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:19.121848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.306038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b9b65-0487-46ad-9508-b2ea15348d4c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:19.367925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c3a89-2c52-41d9-92ea-4d1fd1711cc2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:19.380806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:19.397076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:19.402572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.402572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.402572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.405177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:19.406677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8875a1-0148-4f94-acce-475053efe839 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:19.408722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 75/151] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:19.409899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef238a5-e7fc-4c9e-8359-4da064fffb37 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.410183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef238a5-e7fc-4c9e-8359-4da064fffb37 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.410460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef238a5-e7fc-4c9e-8359-4da064fffb37 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.411914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 76/152] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:19.417086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.417373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.417653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.422009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:19.422643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.422921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.423189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.423971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:19.424277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:19.425791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:19.440077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 77/153] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:19.441151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.461402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:19.461936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:19.463004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:19.592864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 77/154] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:19.593937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.628928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc18a0-3aa6-4343-b03a-fe6e54070f81 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:19.692559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:19.693469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.693899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.694350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.695299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:19.695756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:19.697893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:19.707157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 78/155] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=5a1e89be85c547438247b00c04c09046 => generated 610 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:19.709706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.780401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72dcbe-78be-4a2f-8d5d-5cc1dd32fbc1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:19.832864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:19.865895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:19.865895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:19.865895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:19.865895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] PATH_INFO: `/v3/users/b36e39c714814374bc4c6b1cee0be089` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:19.866485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] RBAC: Authorizing `identity:get_user(user_id=b36e39c714814374bc4c6b1cee0be089)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:19.866485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:19.866485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 78/156] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/users/b36e39c714814374bc4c6b1cee0be089 => generated 290 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:19.866700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:19.866781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:19.926365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d73c4-8ca9-4460-96d9-0b2eb6195674 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.008952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa910-4293-4fa2-acab-9f4c9a9544bd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.037176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.038152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.038513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.038865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.059922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.061120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.061549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.061963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.063377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:20.065019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:20.066942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 79/157] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:20.070185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:20.075780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:20.076678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:20.077431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 79/158] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:20.082289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:20.085739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:20.180863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:20.227090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcfa2e-ac32-454b-942c-3baf18237dc5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.282011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.282710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.282995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.283274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/b36e39c714814374bc4c6b1cee0be089/roles/a6b6bec99f8947f2ac388c27d6e1fd89` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.285315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, user_id=b36e39c714814374bc4c6b1cee0be089, role_id=a6b6bec99f8947f2ac388c27d6e1fd89)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:20.286759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:20.310621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfd3d-dea4-4dc6-b3ae-c32270cbabb2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.345418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:20.351323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.352147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.352766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.353264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.354220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:20.354747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:20.356455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:20.368610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 80/159] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:20 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:20.373926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:20.378847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 80/160] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:20 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/users/b36e39c714814374bc4c6b1cee0be089/roles/a6b6bec99f8947f2ac388c27d6e1fd89 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:20.517206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ad83-9b40-40ae-86cf-b89a581eb91c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.558560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.559217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.559499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.559776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.564278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:20.565783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c84b0-afce-4c9d-87fd-01999b0b8f27 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:20.567856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 81/161] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:20 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:20.573746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:20.695131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcd54-2b0f-4a17-bec0-29cec8e286fa None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.695458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcd54-2b0f-4a17-bec0-29cec8e286fa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.695740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcd54-2b0f-4a17-bec0-29cec8e286fa None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.697194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 82/162] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:20.702305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.702590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.702868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.718274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd698-964a-46ba-baa2-b344caa395a7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:20.733425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:20.734660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:20.776838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:20.782705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:20.783346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:20.783629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:20.783910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:20.867924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:20.869696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:20.881658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:20.909847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 83/163] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 16:13:20.950709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:20.980075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 81/164] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:20.994727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:21.048829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:21.126510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcecc6-ed5b-4c7b-971a-1a34bfe6e32e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:21.198275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b6836-b859-4ce4-b61b-305be9073c3a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:21.216039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:21.216954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.217400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.217817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.239047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:21.244420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:21.245455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:21.246406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:21.255520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 84/165] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:21.262209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:21.281454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:21.282408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.282861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.283276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] PATH_INFO: `/v3/projects/8b50b2d802354944913b873c17f2b5bc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.286063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] RBAC: Authorizing `identity:get_project(project_id=8b50b2d802354944913b873c17f2b5bc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:21.288694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:21.292260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 82/166] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:20 2022] GET /identity/v3/projects/8b50b2d802354944913b873c17f2b5bc => generated 313 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:21.299663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:21.388958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:21.511722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e8a1-a88b-4111-a4f6-9abda8343e36 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:21.517931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:21.533324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ae995-9f00-4063-936a-69f4d4ebd06e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:21.559638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:21.560914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.561361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.561967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.562853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:21.563364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:21.565468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:21.575093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 85/167] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:21 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:21.582244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:21.606216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:21.606852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.607135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.607412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.608219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=8b50b2d802354944913b873c17f2b5bc, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:21.608557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:21.610010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b083b64-8d34-4f75-bde0-ec0c1f1e2042 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:21.624568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 83/168] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:21 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=8b50b2d802354944913b873c17f2b5bc&role.id=5a1e89be85c547438247b00c04c09046 => generated 610 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:21.636707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310525-b45e-409c-a298-73b27ce787b0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.637013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310525-b45e-409c-a298-73b27ce787b0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.637276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310525-b45e-409c-a298-73b27ce787b0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.638701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 84/169] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:21.644918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df86e1-6bab-4a83-bb7e-07857ae290d1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df86e1-6bab-4a83-bb7e-07857ae290d1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df86e1-6bab-4a83-bb7e-07857ae290d1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 85/170] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7e14-76b6-4e2e-9155-a8b784331a1d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7e14-76b6-4e2e-9155-a8b784331a1d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.653696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7e14-76b6-4e2e-9155-a8b784331a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.676327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ac3b7e14-76b6-4e2e-9155-a8b784331a1d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:21.677641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ac3b7e14-76b6-4e2e-9155-a8b784331a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:21.803958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171f31-448e-4e68-b4dd-8166f142bf82 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:21.822156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:21.850339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:21.852786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.853230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.854022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] PATH_INFO: `/v3/users/7806a5c324534a3fa0c5c442f35fb401` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.856672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 86/171] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:21.857664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] RBAC: Authorizing `identity:get_user(user_id=7806a5c324534a3fa0c5c442f35fb401)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:21.859319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:21.860975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:21.862950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 86/172] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:21 2022] GET /identity/v3/users/7806a5c324534a3fa0c5c442f35fb401 => generated 276 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:21.863510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:21.863953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:21.872426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:21.895695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:21.898960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:22.026678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 87/173] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:22.027754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.044061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca181-379e-4dd4-a38c-d5ea5674f77d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.071927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.133270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.134929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.135597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.136206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.194844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bac0ad-336f-4d30-896b-d121b6632424 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.226135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.232349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.237629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.237629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.237629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.247208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:22.250217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.251929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c290fb-6cfd-4686-9465-421a58a39e14 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:22.253952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:22.254994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 88/174] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:22.256678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.343108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.367002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 87/175] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:21 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:22.367815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55008-7d72-4ed2-99c7-6ae528d221a0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.368605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a76bea-fd68-4d31-aa75-3a7152714f8c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.368997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a76bea-fd68-4d31-aa75-3a7152714f8c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.369397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a76bea-fd68-4d31-aa75-3a7152714f8c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.370979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 88/176] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:22.372057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6db87-691d-4254-852d-f88187156d52 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.372995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6db87-691d-4254-852d-f88187156d52 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.372995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6db87-691d-4254-852d-f88187156d52 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.376465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 89/177] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:22.376465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6db87-691d-4254-852d-f88187156d52 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.419896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.420485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.420765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.421042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.431341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:22.436488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:22.437317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:22.438528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 89/178] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:22.439485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.477017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.497252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6db87-691d-4254-852d-f88187156d52 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.535994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.536645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.536957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.537232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.537951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:22.538258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.539632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f954278-0c3a-4b18-b879-09daae9992f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.552660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 90/179] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:22.552660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.552660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.552660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.559258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896100-b4b5-4ac9-bbe3-ed738854e2d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.570679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:22.572017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:22.596501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.597134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.597418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.597682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.598834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.600295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a384f-b849-4ee0-a818-ff0a69c09fca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.602227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 90/180] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:22.607866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.608155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.609096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.621412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.627163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:22.628021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:22.642121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 91/181] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:22.650349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.704594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 91/182] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:22.704594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.745950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.772954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577f774-0d9e-4f21-9668-41fecba409a8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.805597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ceb99-24d4-4caa-93ab-71da53c1a8e8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.808845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.809464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.809746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.810018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.811006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:22.811312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.812726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.826409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 92/183] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:22.826409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.845521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.846162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.846441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.846712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] PATH_INFO: `/v3/users/d88063924d80461f80d0ee08a35cdcfc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.847889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] RBAC: Authorizing `identity:get_user(user_id=d88063924d80461f80d0ee08a35cdcfc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.849384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.850978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 92/184] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/users/d88063924d80461f80d0ee08a35cdcfc => generated 248 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:22.851735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:22.855567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.947596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51c47c-1fa1-43c3-af22-b5659f4a991b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.957527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8b3b-5f92-40e3-aa29-a90d6e7901e2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:22.986711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.987393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.987604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.987874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.988704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=7806a5c324534a3fa0c5c442f35fb401) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:22.989016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:22.990996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:22.994293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:22.994559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:22.995583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:22.995927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:22.996184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:22.998007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 93/185] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/role_assignments?user.id=7806a5c324534a3fa0c5c442f35fb401&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 610 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:22.998781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.021970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:23.031961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:23.032917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:23.033689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:23.036600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 93/186] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:23.037748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.121211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.130326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0243e3b-3e61-499e-8f72-e20a82a73629 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.176258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.176934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.177035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b3c-48ef-4146-b91e-d52c74c22242 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.177216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.178032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.179195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.180673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.182264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 94/187] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:22 2022] GET /identity/v3/domains/default => generated 200 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:23.183033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.217769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.218483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.218767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.219038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.238072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.239548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.241932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 94/188] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:23.242725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.276075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.311964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433822f-f64b-4591-9066-ec8659c041e3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.346103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.346724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.346995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.347264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.350846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae5c3-dfc3-4f65-81a9-5a0eeeb748f3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.373005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.384199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.384840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.385118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.385386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.386134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:23.386427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.387787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.395435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 95/189] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:23.396296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.415830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.417320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.484630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.506389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcb67-c33f-470a-8887-69548653d7ac None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.520861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 95/190] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:23.526101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.526943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.528543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.528826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.529101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.529786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.531220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055bdd7-a0e0-4063-8844-08937a527f6c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.621571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 96/191] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:23 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:23.622818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1bf63-47f4-43ca-b992-92f0ccf37cb6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.623105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1bf63-47f4-43ca-b992-92f0ccf37cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.623381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1bf63-47f4-43ca-b992-92f0ccf37cb6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.624656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 97/192] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:23.625369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1bf63-47f4-43ca-b992-92f0ccf37cb6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.634996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89256827-a3f4-4de7-9d76-81fff0189f84 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.672646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.673286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.673564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.673840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.686043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.714135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.751229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.751408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1bf63-47f4-43ca-b992-92f0ccf37cb6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.752688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dd724-01b2-4c97-ab7c-ebf496be64ac None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.791323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.791965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.792342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.792702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.795965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.796952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.798778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 98/193] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:23.799363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.877364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 96/194] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:23.879387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928722e4-c853-4262-936c-2c29e6314239 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.879841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928722e4-c853-4262-936c-2c29e6314239 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.880127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928722e4-c853-4262-936c-2c29e6314239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.906124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-928722e4-c853-4262-936c-2c29e6314239 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:23.907295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-928722e4-c853-4262-936c-2c29e6314239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:23.925890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d535c-9fe9-446c-9d13-3b3e9e16d46b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:23.937003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.970320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:23.971071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:23.971420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:23.971803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:23.973927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:23.975988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:23.976250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:23.978678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 99/195] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:23.979442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:23.988651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 97/196] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:23.990493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928722e4-c853-4262-936c-2c29e6314239 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.104986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7486b-a591-4790-8d49-9d5ba66537a8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.121466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928722e4-c853-4262-936c-2c29e6314239 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.140224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.140899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.141179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.141452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.160326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.161178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.161525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.161923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.163158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.164793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.166704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 98/197] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:24.167574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.189428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:24.215290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.216917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.251768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:24.288967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62327c-697e-4b38-af29-b7ee78176bdf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.328195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 100/198] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:24.332175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.332849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.333135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.333406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.333506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.334753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=d88063924d80461f80d0ee08a35cdcfc) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:24.335064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.336873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.344312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 99/199] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/role_assignments?user.id=d88063924d80461f80d0ee08a35cdcfc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:24.349379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.428641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:24.453906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f4f3-3a74-45e0-a10b-d6bf88a2f9e6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.494803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.495639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.495917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.496185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.508737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:24.514114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:24.514980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:24.515758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 101/200] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:24.516380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.625270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606100-39f1-4549-94c6-cb35ce2412e3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.642153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f17d0-0b0d-4ec8-91a1-15e922202cd0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.668286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:24.674500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.675122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.675736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.676008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.677361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=5d81ca9f5ede46f7b68716ab2690c281) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:24.677831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.681575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.683453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.684197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.684524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.684796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.686299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.686405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 100/201] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/role_assignments?user.id=5d81ca9f5ede46f7b68716ab2690c281&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:24.687252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.688099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.690188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 102/202] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:24.697232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:24.702573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:24.883611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d370-0d91-4b3f-9544-e4fccfc90185 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.936883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:24.937427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:24.937917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:24.938018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:24.939424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:24.939424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:24.940343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:24.949540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 101/203] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:24.968632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc587-83d5-4b63-8f32-96aab0adeb85 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:24.969804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:25.008667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:25.009545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:25.009972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:25.010382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:25.011486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=5d81ca9f5ede46f7b68716ab2690c281) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:25.011965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:25.013618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:25.019885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 103/204] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/role_assignments?user.id=5d81ca9f5ede46f7b68716ab2690c281&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 260 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:25.074669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:25.134109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf20e-d4b2-4f9f-a7cd-2882bed4ca57 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:25.183219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:25.183875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:25.184158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:25.184465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] PATH_INFO: `/v3/users/b36e39c714814374bc4c6b1cee0be089` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:25.185896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] RBAC: Authorizing `identity:get_user(user_id=b36e39c714814374bc4c6b1cee0be089)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:25.187416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:25.189008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 102/205] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:24 2022] GET /identity/v3/users/b36e39c714814374bc4c6b1cee0be089 => generated 290 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:25.195553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:25.315214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:25.356036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166884c-682f-4ebb-a676-5998ea46ca15 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:25.402949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:25.403576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:25.403854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:25.404129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:25.486975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:25.488368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:25.572540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:25.611274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 104/206] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:25.621091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:25.772778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf965-06fa-4bb1-92e1-1b23a967c2b4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:25.812763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:25.826389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:25.827030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:25.827308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:25.827581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:25.833119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:25.835122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e0cfe-ccf7-49b8-853d-ad1fdb4e4a75 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:25.837076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 103/207] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:25 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:25.843072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.001100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879daf2-e76e-4490-940f-db78d50a3cfb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.035426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.035971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.036247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.036598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.037399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c, user.id=b36e39c714814374bc4c6b1cee0be089) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:26.037707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:26.039173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e5c15-33eb-4428-99d6-932ebadf0459 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:26.045270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 105/208] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:25 2022] GET /identity/v3/role_assignments?user.id=b36e39c714814374bc4c6b1cee0be089&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 610 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:26.058214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a57b45-463e-4090-bedd-e9d11f069825 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.058541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a57b45-463e-4090-bedd-e9d11f069825 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.058780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a57b45-463e-4090-bedd-e9d11f069825 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.060433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 104/209] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:26.066042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e9f5a-5e37-4c08-8bfe-163032cad57f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.066387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e9f5a-5e37-4c08-8bfe-163032cad57f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.066673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e9f5a-5e37-4c08-8bfe-163032cad57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.086254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-660e9f5a-5e37-4c08-8bfe-163032cad57f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:26.087482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-660e9f5a-5e37-4c08-8bfe-163032cad57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:26.114547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:26.158554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 106/210] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:26.181749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a57b45-463e-4090-bedd-e9d11f069825 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.220022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921253bf-18bf-4bc3-b86f-848fd176b211 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.220363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921253bf-18bf-4bc3-b86f-848fd176b211 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.220608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921253bf-18bf-4bc3-b86f-848fd176b211 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.222105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 107/211] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:26.228028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.228327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.228649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.257973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:26.259357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:26.354761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a57b45-463e-4090-bedd-e9d11f069825 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.357213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 108/212] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:26.384242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.389633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.390280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.390560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.390833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.402001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:26.407914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:26.408777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:26.409554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 105/213] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:26.415686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.481135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:26.488448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:26.500194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abe8dd-3caf-4cfb-8bfc-e7cadab7eada None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.550647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.551879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e977-5b23-4091-ad85-ce89de5e703a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.552483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.552796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.553076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.555752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:26.557283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:26.559112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 109/214] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:26.566998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.606714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.607273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.607552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.607824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.608596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:26.608904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:26.610354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:26.621518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 106/215] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:26 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:26.628736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.709218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfab36-5a5a-4695-a72c-e4c791256d8a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.746129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.746850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.747205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.747555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.748309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:26.749842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f60ed4-86ad-4861-ad14-8ba1a9952881 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:26.752885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fb82f-2116-4fbb-a88a-97d4f4ca0c3f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.768342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:26.788880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.789555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.789804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.790098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] PATH_INFO: `/v3/users/df74895dc83343a3b2be336524b2121d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:26.803466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] RBAC: Authorizing `identity:get_user(user_id=df74895dc83343a3b2be336524b2121d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:26.805041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:26.809639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 107/216] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:26 2022] GET /identity/v3/users/df74895dc83343a3b2be336524b2121d => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:26.812598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:26.844098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 110/217] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:26 2022] POST /identity/v3/users => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:13:26.943325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd35e50-f5b6-4bba-91e7-a605ed315e42 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:26.984095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:26.984772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:26.985053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:26.985381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.032091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.056478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492864cc-4a62-4943-9728-0494656dec59 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.056478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492864cc-4a62-4943-9728-0494656dec59 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.056478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492864cc-4a62-4943-9728-0494656dec59 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.057881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 111/218] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:27.063487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:27.063714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.064105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.064317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.065071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:27.088571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:27.089759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:27.102358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.180701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 112/219] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:27.195805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 108/220] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:26 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:13:27.202326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.205070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.300745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.318832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12117d59-0e95-4c0c-9e5a-5618ba3b817b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.325083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96cbc1-107b-49dc-a977-1fe16dd0c426 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.357189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:27.358217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.359124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.359621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.361801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:27.363535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:27.366091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 113/221] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:27.372313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.374076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:27.374727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.375013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.375290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.392424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:27.398151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:27.398990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:27.399757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:27.400331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.402197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 109/222] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:27.413500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.531665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780d91-daee-4e23-91c8-59b41e7c565a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.560506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae0e70-80e9-4120-80b3-986ef3fa1ad1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.568511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:27.569678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.569960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.570234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.571509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=df74895dc83343a3b2be336524b2121d) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:27.571827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:27.574027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c737551-1335-469a-9ba4-9bdeb9d7640d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:27.581551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 114/223] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/role_assignments?user.id=df74895dc83343a3b2be336524b2121d&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=5a1e89be85c547438247b00c04c09046 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:27.584765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8839-0478-46fd-ba47-20064af7b52a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.585069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8839-0478-46fd-ba47-20064af7b52a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.585353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8839-0478-46fd-ba47-20064af7b52a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.586749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 115/224] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:27 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:27.599691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.600076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.600361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.607150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.620929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:27.622157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:27.629181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:27.629951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.630361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.631261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.632084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:27.632507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:27.634013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:27.644709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 110/225] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:27.654302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.769624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 116/226] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:27.806449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.818638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:27.842827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b67e-0c34-45e9-8137-05cc49b2b25b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.913304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:27.914243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:27.916933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:27.917387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:27.919614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:27.921249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:27.924228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 111/227] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:27.930484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:27.979024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdb904-fb59-4867-9bef-5dadbc8a7cae None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:27.995989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:28.016502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.020659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.020659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.020659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.028580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:28.033485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:28.034360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:28.035344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 117/228] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:28.040061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090117f-37f5-4e0e-978d-4db90e641cda None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.042465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:28.085122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.086152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.086647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.087109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.089068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.092953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.101688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 112/229] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:27 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:28.105746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:28.180829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:28.223656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e0fa9-2cb7-4642-bb23-5abbc92f26a9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.306086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df939-4da3-4464-acb8-bff3f20771a4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.309783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.310668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.310950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.311224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.311957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:28.312266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.313688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.330574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 118/230] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:28 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:28.336118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:28.340754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:28.354179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.355137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.356179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.356179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/5d81ca9f5ede46f7b68716ab2690c281/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.359740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, user_id=5d81ca9f5ede46f7b68716ab2690c281, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.362798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.467334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 113/231] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:28 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/5d81ca9f5ede46f7b68716ab2690c281/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:28.476323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c900-fc73-4397-a7c9-d37b67bf1513 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.527076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:28.535530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.536263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.537097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.537368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.538588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.540379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.542055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 119/232] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:28 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:28.550097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:28.686600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fb6f-d4ba-457a-a9d1-18438987a216 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.722732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.723393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.723679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.724544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.725812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.727360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b0380-ef69-42a3-a170-6610f7913277 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.729455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 114/233] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:28 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:28.738493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:28.860856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02106353-46d0-41d7-b235-ed0c9ac9c7b2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:28.894545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:28.895210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:28.895486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:28.895758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/5d81ca9f5ede46f7b68716ab2690c281/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:28.899989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, user_id=5d81ca9f5ede46f7b68716ab2690c281, role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:28.901461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438160-11b5-49d2-845e-3f349a041f50 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:28.999350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 120/234] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:28 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/5d81ca9f5ede46f7b68716ab2690c281/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:29.522438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9d4-c449-49dd-897e-7865a11a0e68 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:29.523221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9d4-c449-49dd-897e-7865a11a0e68 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:29.523221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9d4-c449-49dd-897e-7865a11a0e68 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:29.524604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 115/235] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:29.529960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e24b-1c7d-445c-952c-61be49c81b54 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:29.530610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e24b-1c7d-445c-952c-61be49c81b54 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:29.530610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e24b-1c7d-445c-952c-61be49c81b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:29.557951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-59f2e24b-1c7d-445c-952c-61be49c81b54 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:29.559163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-59f2e24b-1c7d-445c-952c-61be49c81b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:29.640684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:29.660159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 121/236] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:29.721552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52e9d4-c449-49dd-897e-7865a11a0e68 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:29.796615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:29.839336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ac623-6d5e-4843-855e-a056401a0829 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:29.839627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ac623-6d5e-4843-855e-a056401a0829 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:29.839913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ac623-6d5e-4843-855e-a056401a0829 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:29.841423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52e9d4-c449-49dd-897e-7865a11a0e68 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:29.841559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 122/237] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:29.849763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea8b-d200-482a-aca7-3ea4731c6394 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:29.849763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea8b-d200-482a-aca7-3ea4731c6394 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:29.849975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea8b-d200-482a-aca7-3ea4731c6394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:29.877535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:29.878454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:29.878892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:29.879308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:29.892042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7e70ea8b-d200-482a-aca7-3ea4731c6394 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:29.892708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:29.893356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e70ea8b-d200-482a-aca7-3ea4731c6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:29.898193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:29.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:29.900041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:29.901962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 116/238] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:29.912495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.043792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdb91-7674-49f1-afa6-9b2616a88f84 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.070789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 123/239] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:30.072134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.093370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.093992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.094274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.094547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.095273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:30.095577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.097373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3420f-1c49-427f-900e-9d7613c3d11e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.108536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e8c3-8f90-4d42-92eb-1c864e1825e7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.108829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e8c3-8f90-4d42-92eb-1c864e1825e7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.109101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e8c3-8f90-4d42-92eb-1c864e1825e7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.110846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 124/240] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:30.111953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e8c3-8f90-4d42-92eb-1c864e1825e7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.116521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 117/241] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:29 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:30.122049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.122571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.123051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.143566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:30.145105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:30.219977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 118/242] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:30.225676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e8c3-8f90-4d42-92eb-1c864e1825e7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.226952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.244640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.270193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.271056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.271437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.271831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.273494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.276843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 125/243] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:30.277718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.353512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b554e-aa5d-46ac-b530-6ba325402f68 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.365227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.387766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f8e7c-4261-441f-a569-5e410f251c09 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.389534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.390344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.391261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.391261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] PATH_INFO: `/v3/users/4769def4910e40e6a71a822341a9f9a7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.405224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] RBAC: Authorizing `identity:get_user(user_id=4769def4910e40e6a71a822341a9f9a7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.406803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.408841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 119/244] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/users/4769def4910e40e6a71a822341a9f9a7 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:30.409560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.425896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.426730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.427694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.428131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.439617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:30.446159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:30.447011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:30.447758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:30.450159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 126/245] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:30.455702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.514548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7525d-bbbd-45ca-bf25-a5d4e3e3d480 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.524773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.548526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.552616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.552616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.552616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.552616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.552616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.576062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de252700-602f-4141-99d4-5d91cfb2abb4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.611523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.612241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.612643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.612932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.635663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 120/246] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:30 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:30.643806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.711128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.745407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.746897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.819985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ceb13-979d-4a8f-9c4f-dab2be2dfae5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:30.821700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:30.858070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:30.859079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:30.859560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:30.860881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:30.862198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:30.862795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:30.864503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:30.874617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 121/247] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:30.882201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.886426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 127/248] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:30.892060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:30.980827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:31.003844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6b18e-95f5-4fd3-bfc4-272b38603ccb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.004859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f7d-51fa-4bff-b596-6420413f4c45 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.042507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.043157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.043438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.043713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.044326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.044911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.045532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.046010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.046484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.046587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] PATH_INFO: `/v3/users/df74895dc83343a3b2be336524b2121d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.047983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] RBAC: Authorizing `identity:get_user(user_id=df74895dc83343a3b2be336524b2121d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.049370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 128/249] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:31.050319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.053083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 122/250] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:30 2022] GET /identity/v3/users/df74895dc83343a3b2be336524b2121d => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:31.059228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.062364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.077008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:31.190489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3c6a-2837-41f5-a1e4-24ad05c6873b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.195246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6421d29-eb68-4fef-8687-5e4b8c55013c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.235869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.236601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.236883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.237164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.238380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.239174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.240370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3583d7-6090-400d-accb-1bf7d7a2de7b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.240792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.241265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.241732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.242389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 129/251] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:31.242937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=4769def4910e40e6a71a822341a9f9a7) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:31.243473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.243573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89747-b1b9-438e-8686-841fddff654e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.243813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89747-b1b9-438e-8686-841fddff654e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.244090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89747-b1b9-438e-8686-841fddff654e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.245715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9ecb9-320a-4f4c-bb1a-4a92da204f75 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.249642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 130/252] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:31 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:31.251765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e89747-b1b9-438e-8686-841fddff654e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.252732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 123/253] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/role_assignments?user.id=4769def4910e40e6a71a822341a9f9a7&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:31.256322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.256875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.257390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.280124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:31.281399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:31.298257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:31.367446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 124/254] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:31.372844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:31.391519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e89747-b1b9-438e-8686-841fddff654e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.393661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.430651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.431342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.431622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.431894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/df74895dc83343a3b2be336524b2121d/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.434325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, user_id=df74895dc83343a3b2be336524b2121d, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.436441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.519060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 131/255] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:31 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/df74895dc83343a3b2be336524b2121d/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:13:31.533797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb072205-8a12-4bf2-939e-d903b3def53f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.578136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.578798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.579076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.579349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.591755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-51ec4bbb-83a6-4a30-84f5-92f5053cda8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:31.596759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:31.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:31.599386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 125/256] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:31.604435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.742215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd26da-d253-46d7-96be-97c26d7c66a8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.742541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd26da-d253-46d7-96be-97c26d7c66a8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.742774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd26da-d253-46d7-96be-97c26d7c66a8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.744468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 126/257] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:31.751532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.751834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.752079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.784865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:31.786092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:31.788845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:31.800451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a5ce7-40cf-4072-8dbd-0ec3f22599ff None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:31.872926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:31.873557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:31.873836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:31.874106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:31.874827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:31.875130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:31.878839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 127/258] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 16:13:31.878956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:31.892515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 132/259] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:31.898788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.911994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:31.918183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.011759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442bff-701b-4250-a16d-01fc7d33b703 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.038000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddcff5-1a75-43f1-9d14-458a07939e43 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.047528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.048181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.048490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.048772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.049949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.051403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.053583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 128/260] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:32.058867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.068726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.097082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.098071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.098572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.099609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.113287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:32.125150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:32.126116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:32.126869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:32.129583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 133/261] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:32.134903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.163173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77cc71-9318-4d0d-ac71-845662c61926 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.198052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.198679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.198964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.199241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.236815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.268237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f2877-6e61-48bc-bd1f-d64151a6f96a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.292597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.294954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.303929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.304738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.305134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.305499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.305757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.306599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:32.306806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.308468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.318011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 134/262] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:32.323979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.469316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f619e9-3dba-4cbd-bd76-530f5c5736f0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.501674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 129/263] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:32.504420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.508129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.512853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.515250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.515250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.515250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] PATH_INFO: `/v3/users/5d81ca9f5ede46f7b68716ab2690c281` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.521494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] RBAC: Authorizing `identity:get_user(user_id=5d81ca9f5ede46f7b68716ab2690c281)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.523158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.525105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 135/264] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/users/5d81ca9f5ede46f7b68716ab2690c281 => generated 284 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:32.531435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.651624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4cdef-9d0a-4c49-94e8-2e880a76da5f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.688670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.688900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83e8d-e078-42a1-91c7-4869f06797aa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.690612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.691091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.691609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.700103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.751160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.752022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.752199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.752512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.757225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.758695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.759557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:32.761837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 130/265] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:32.767140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.768236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.769982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:32.881451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 136/266] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:32.890028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:32.942340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99437-014a-48cc-b0c6-a4d5e967c6f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:32.995672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:32.996284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:32.996587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:32.996862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:32.997660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=5d81ca9f5ede46f7b68716ab2690c281) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:32.997973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:32.999403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:33.019867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 131/267] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/role_assignments?user.id=5d81ca9f5ede46f7b68716ab2690c281&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:33.041162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:33.050243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce2e3b-4357-4ab9-9eb3-c1e615acccc4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:33.088046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:33.089375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:33.089882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:33.090346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:33.091873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:33.093553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9acd-8d10-4b86-b642-6512392dcaac None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:33.095414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 137/268] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:32 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:33.105517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:33.152133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:33.217655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204643e-a431-4d27-ba1e-00f43f1d4739 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:33.254073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:33.254705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:33.255513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:33.255788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:33.256664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=5d81ca9f5ede46f7b68716ab2690c281) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:33.256977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:33.258450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b166c9-65c2-4854-9e9c-f6bf99545832 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:33.265087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 132/269] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:33 2022] GET /identity/v3/role_assignments?user.id=5d81ca9f5ede46f7b68716ab2690c281&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 610 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:33.960549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cac7-8626-480a-bf20-7a35bbccfe52 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:33.961066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cac7-8626-480a-bf20-7a35bbccfe52 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:33.961138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cac7-8626-480a-bf20-7a35bbccfe52 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:33.962455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 138/270] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:33.967720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:33.968115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:33.968276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:33.987060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:33.988232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:33.996646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3694-09cd-475a-be7b-e26754c69d18 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:33.996936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3694-09cd-475a-be7b-e26754c69d18 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:33.997213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3694-09cd-475a-be7b-e26754c69d18 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:33.998551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 139/271] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:34.003298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.003586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.003864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.024135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:34.026151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:34.059116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 133/272] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:34.081663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.111013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 140/273] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:34.122111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.128805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.210396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea6f98-a948-4988-8209-582a9bc1b148 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.240133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.250200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd49cc-bc41-4470-b563-64353eb57cfd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.250859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.251508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.251788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.252059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.266096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:34.273158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:34.274211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:34.275180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:34.277431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 134/274] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:34.283285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.296977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.297495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.297852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.298201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.310596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-512d1e89-6753-4f0e-a65e-be0cbc30b9b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:34.318187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:34.319110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:34.319860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:34.321027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 141/275] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:34.321754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cfad-25be-42fb-8f8b-f4834362b7ec None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.322035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cfad-25be-42fb-8f8b-f4834362b7ec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.322308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cfad-25be-42fb-8f8b-f4834362b7ec None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.323572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 142/276] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:34 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:34.325817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11cfad-25be-42fb-8f8b-f4834362b7ec None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.391168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.401205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d3d2-f12c-437c-b873-137c0fa1011d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.437701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.438247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.438529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.438957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.439514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:34.439807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.441298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea12fb-3f1e-4ada-a10c-f568b136816a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.442762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11cfad-25be-42fb-8f8b-f4834362b7ec None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.453936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 135/277] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:34.453936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.453936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.453936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.470468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:34.471658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:34.485702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.486324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.486603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.486872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.487588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:34.487941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.492009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.506226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 143/278] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.506226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.520097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.551492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 136/279] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:34.552934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.613463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e09e7a-0d13-43ba-8da0-8b42dbcf307d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.647690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.650567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.651204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.651811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.651811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] PATH_INFO: `/v3/users/9fce6b634ae74c4c91ffd8f7e2508613` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.662911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b50dd0-49e1-46de-9568-40a79e73056b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.665455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=9fce6b634ae74c4c91ffd8f7e2508613)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.666904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.668568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 144/280] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/users/9fce6b634ae74c4c91ffd8f7e2508613 => generated 294 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.669336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.698646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.699277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.699732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.700326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] PATH_INFO: `/v3/users/4769def4910e40e6a71a822341a9f9a7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.702001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] RBAC: Authorizing `identity:get_user(user_id=4769def4910e40e6a71a822341a9f9a7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.703425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31699e-855b-4481-bb47-2f5f1092cf47 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.705083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 137/281] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/users/4769def4910e40e6a71a822341a9f9a7 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.706317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff067-2c50-46fb-b229-b81afc159b5f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.706599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff067-2c50-46fb-b229-b81afc159b5f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.706905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff067-2c50-46fb-b229-b81afc159b5f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.712535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 138/282] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:34.712535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff067-2c50-46fb-b229-b81afc159b5f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.740589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.768980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013487-1961-43a2-b1e4-9e7d14b0a5c3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.806398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.807037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.807331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.807594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.808780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.810489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.811584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff067-2c50-46fb-b229-b81afc159b5f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.812071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 145/283] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.818085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.847833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.848461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.848771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.849045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.849831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, user.id=9fce6b634ae74c4c91ffd8f7e2508613, scope.system=all) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:34.850141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.851574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031c3c5-6c12-4bd2-9e0c-c53b9664a0de None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.861791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 139/284] 10.222.0.144 () {56 vars in 1313 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/role_assignments?user.id=9fce6b634ae74c4c91ffd8f7e2508613&scope.system=all&role.id=5a1e89be85c547438247b00c04c09046 => generated 227 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.861791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.861791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.861791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.882512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:34.883760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:34.895946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:34.921059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76e7e4-db98-42b4-933a-a3c24f08956f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:34.959189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 140/285] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:34.960040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.962537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:34.963295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:34.963649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:34.963993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:34.965317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:34.966868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:34.968955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 146/286] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:34 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:34.974797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:34.987952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:35.082929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf22063-d038-49a2-99b4-661c0b92cb00 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.088936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b7bc-f522-4b38-a77b-50648a47c906 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.124865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.125487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.125878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.126150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:35.126947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:35.128680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:35.131758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.132446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.132726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.132995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/4769def4910e40e6a71a822341a9f9a7/roles/a6b6bec99f8947f2ac388c27d6e1fd89` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:35.134912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, user_id=4769def4910e40e6a71a822341a9f9a7, role_id=a6b6bec99f8947f2ac388c27d6e1fd89)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:35.136349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:35.199645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 147/287] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:13:34 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/users/4769def4910e40e6a71a822341a9f9a7/roles/a6b6bec99f8947f2ac388c27d6e1fd89 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:13:35.200266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:35.221518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 141/288] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:13:34 2022] POST /identity/v3/users => generated 294 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:35.280974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:35.327570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b498-fe6b-4f22-93b0-ca15847212a5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.371449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.372116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.372410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.372713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:35.385527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:35.390979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:35.391893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:35.392676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:35.396812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 148/289] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:35.398999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:35.402138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:35.516894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602156d-4cd4-4209-b966-f1395882a48d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.555044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.555669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.555948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.556221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:35.557043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:35.557346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:35.558709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:35.569328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 142/290] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:35 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:35.574973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:35.696642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5419c5-94d1-4afb-88c8-9271259f6193 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.725267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:35.734790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.735695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.736167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.736650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] PATH_INFO: `/v3/users/df74895dc83343a3b2be336524b2121d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:35.738019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] RBAC: Authorizing `identity:get_user(user_id=df74895dc83343a3b2be336524b2121d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:35.739756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:35.743723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 149/291] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:35 2022] GET /identity/v3/users/df74895dc83343a3b2be336524b2121d => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:35.749145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:35.837978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:35.885017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53d73-e0df-4a0a-8ddc-5b18c79f7f8c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:35.937119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:35.937750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:35.938037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:35.938319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:36.076146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:36.077982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:36.147035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:36.186578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 143/292] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:36.192257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:36.325622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52102d-c316-4d14-b1f8-12947aaf7660 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:36.368034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:36.368699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:36.368976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:36.369248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:36.373072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:36.374526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519518e-443a-4444-9a70-ecec7f47a5c4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:36.376168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 150/293] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:36 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:36.382566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:36.540409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7feaa-4198-41db-b336-e12e8cd5c265 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:36.596678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:36.597305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:36.597584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:36.597854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:36.599256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=df74895dc83343a3b2be336524b2121d) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:36.599572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:36.601034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9974aae-6b38-4127-8dc2-2a1cabec14e5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:36.613561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 144/294] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:36 2022] GET /identity/v3/role_assignments?user.id=df74895dc83343a3b2be336524b2121d&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=5a1e89be85c547438247b00c04c09046 => generated 610 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:37.238333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008f3d-7a83-4d46-b78d-87907aa26b44 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.238862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008f3d-7a83-4d46-b78d-87907aa26b44 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.238862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008f3d-7a83-4d46-b78d-87907aa26b44 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.241059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 151/295] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:37.246078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.246486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.246670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.266492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:37.267678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:37.290882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:37.344571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 145/296] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:37.360111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008f3d-7a83-4d46-b78d-87907aa26b44 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:37.427115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:37.467349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008f3d-7a83-4d46-b78d-87907aa26b44 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:37.521915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:37.522554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.522825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.523092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.534096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5acb1d4b-8f03-48a3-9e8c-0eb84f1f9320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:37.539527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:37.540369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:37.541136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 152/297] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:37.546388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:37.646902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838243e-f9cf-41a0-8d78-b94a07dc394a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.647215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838243e-f9cf-41a0-8d78-b94a07dc394a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.647453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838243e-f9cf-41a0-8d78-b94a07dc394a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.649096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 153/298] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:37.661906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12c64-8bd3-45f7-a084-254d6898a0bf None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.661906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12c64-8bd3-45f7-a084-254d6898a0bf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.661906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12c64-8bd3-45f7-a084-254d6898a0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.662382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb6d71-138f-47e2-854e-d9291a38e70b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:37.675426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a8e12c64-8bd3-45f7-a084-254d6898a0bf None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:37.676875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a8e12c64-8bd3-45f7-a084-254d6898a0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:37.691948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548504d7-029e-497a-8325-8743b405976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:37.693144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504d7-029e-497a-8325-8743b405976c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.693426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504d7-029e-497a-8325-8743b405976c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.693701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504d7-029e-497a-8325-8743b405976c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.694425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548504d7-029e-497a-8325-8743b405976c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:37.694736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548504d7-029e-497a-8325-8743b405976c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:37.696115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548504d7-029e-497a-8325-8743b405976c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:37.710002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 146/299] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:37 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:37.711342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352b057-c02b-4b52-9038-29a44c8acc90 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.711635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352b057-c02b-4b52-9038-29a44c8acc90 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.711914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352b057-c02b-4b52-9038-29a44c8acc90 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.713195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 147/300] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:37.713892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352b057-c02b-4b52-9038-29a44c8acc90 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:37.738057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:37.757272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 154/301] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:37.757272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.757272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.757272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.777633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:37.778831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:37.826521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352b057-c02b-4b52-9038-29a44c8acc90 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:37.833046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:37.852002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 155/302] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:37.857102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:37.882783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:37.883494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:37.883790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:37.884070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] PATH_INFO: `/v3/users/9fce6b634ae74c4c91ffd8f7e2508613` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:37.886938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] RBAC: Authorizing `identity:get_user(user_id=9fce6b634ae74c4c91ffd8f7e2508613)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:37.887647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:37.892256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 148/303] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:37 2022] GET /identity/v3/users/9fce6b634ae74c4c91ffd8f7e2508613 => generated 294 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:37.893570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:37.966861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fc1df-5fff-4e3d-a521-dfffc77fed11 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:37.988655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:37.999651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.000181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d248-a48e-4eb9-9aa2-6b7ea7a21987 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.000331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.000591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.000862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.011269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:38.016193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:38.017138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 156/304] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:38.023561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.033846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.034481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.034765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.035041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.045806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f11da527-a905-41ac-8e84-a528f0eefb8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:38.051144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:38.052121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:38.053328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 149/305] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:38.053957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f197-6e35-470d-a870-96bccc932ac9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.053957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f197-6e35-470d-a870-96bccc932ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.054111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f197-6e35-470d-a870-96bccc932ac9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.055347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 150/306] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:38 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:38.056037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4f197-6e35-470d-a870-96bccc932ac9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.069521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.121999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579d55c-48a6-46bd-8309-540953425142 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.155839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.156513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.156794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.157065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] PATH_INFO: `/v3/system/users/9fce6b634ae74c4c91ffd8f7e2508613/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.157340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4f197-6e35-470d-a870-96bccc932ac9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.158519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9fce6b634ae74c4c91ffd8f7e2508613, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.159908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e8768-bcdb-4ab1-a84d-e5f0fdf8edbe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.190883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.191494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.191778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.192052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.193731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:38.194040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.195454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.203816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 151/307] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.206178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.216705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.254469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 157/308] 10.222.0.144 () {58 vars in 1339 bytes} [Tue Jul 26 16:13:38 2022] PUT /identity/v3/system/users/9fce6b634ae74c4c91ffd8f7e2508613/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:38.256260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.256573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.256842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.277147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:38.278412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:38.317067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fc35-59ef-4438-95e1-e939ca40bcfb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.357264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 158/309] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:38.358154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.369676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.370705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.371074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.371417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.372234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:38.372652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.374194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.386939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 152/310] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.388022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.420019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.478441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.481539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f1bb3-34d7-4e96-bad9-aa5fe2a4556a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.518960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b34d9-f658-4b7a-bc45-39c6f6386b67 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.520066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.521413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.521694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.522680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] PATH_INFO: `/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.535297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] RBAC: Authorizing `identity:get_user(user_id=4e80d013c7c14d9e98c1a0565c90ba9b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.536756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.538624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 159/311] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b => generated 294 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.539180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.562035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.562685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.562972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.563250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.564419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.565900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.572579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 153/312] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.572579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.642100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.647913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f7b9e-4169-46c6-b535-b1f061ae652c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.677065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f96ab3-7e76-4dbe-879e-927f31e0887c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.684774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.685679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.685964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.686250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] PATH_INFO: `/v3/users/4769def4910e40e6a71a822341a9f9a7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.687433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] RBAC: Authorizing `identity:get_user(user_id=4769def4910e40e6a71a822341a9f9a7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.688926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.690536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 160/313] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/users/4769def4910e40e6a71a822341a9f9a7 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.691279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.718649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.719262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.719620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.719899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.720720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, user.id=4e80d013c7c14d9e98c1a0565c90ba9b, scope.system=all) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:38.721030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.722452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.727545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 154/314] 10.222.0.144 () {56 vars in 1313 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/role_assignments?user.id=4e80d013c7c14d9e98c1a0565c90ba9b&scope.system=all&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 227 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:38.733434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:38.738567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:38.830978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0aabb-1c91-4d0e-b6a2-4f9a73b4520d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.852885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4a4d-a934-416f-9d2f-009700d9c77e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:38.872178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.873308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.873808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.874280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] PATH_INFO: `/v3/groups` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.875271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.877075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.892919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:38.893543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:38.893824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:38.898051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:38.952101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 161/315] 10.222.0.144 () {60 vars in 1139 bytes} [Tue Jul 26 16:13:38 2022] POST /identity/v3/groups => generated 230 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:38.964265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:38.965867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:38.969352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:39.084804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 155/316] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:38 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:39.090339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:39.124628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:39.204873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb6ce6-301b-4f7b-affa-ed60eb30e90b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:39.241826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:39.242465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:39.242745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:39.243014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:39.244178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:39.245931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f85f51-407d-4a89-aaa4-916ace9bdb2d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:39.247907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 162/317] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:39 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:39.253830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:39.358130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8598b-b630-4566-aebc-3ead5e9e1d2d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:39.367106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:39.394601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:39.395738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:39.396149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:39.396491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:39.397295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc, user.id=4769def4910e40e6a71a822341a9f9a7) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:39.397600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:39.399023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecac02-7491-48b9-ac20-dd2d341fc600 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:39.405621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 156/318] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:13:39 2022] GET /identity/v3/role_assignments?user.id=4769def4910e40e6a71a822341a9f9a7&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:40.245944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397bf9-04f6-4f9f-9b5c-ac0581242714 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.246456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397bf9-04f6-4f9f-9b5c-ac0581242714 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.246530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397bf9-04f6-4f9f-9b5c-ac0581242714 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.247973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 163/319] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:40.252901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.253255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.253597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.272025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:40.273225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:40.338094 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 157/320] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:40.361068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397bf9-04f6-4f9f-9b5c-ac0581242714 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:40.451598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:40.461482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397bf9-04f6-4f9f-9b5c-ac0581242714 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:40.499243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:40.499856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.500141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.500429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.511245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:40.516177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:40.517170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:40.517928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 164/321] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:40.523835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:40.620274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:40.624666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9678-12ce-4cc5-931e-534455f0da0f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:40.659869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:40.660677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.660971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.661256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.662201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:40.663001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:40.664405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b9f11-769b-4ee2-909a-a79594329c59 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:40.674427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 158/322] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:40 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:40.680223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:40.762079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903538-b289-40f8-bbfa-56660c349859 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.762267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903538-b289-40f8-bbfa-56660c349859 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.762551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903538-b289-40f8-bbfa-56660c349859 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.764033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 159/323] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:40.768654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.768947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.769229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.784737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af609a-c2ce-4a67-b35b-2244e8e06466 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:40.788173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:40.789422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:40.825902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:40.826554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.826835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.827118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] PATH_INFO: `/v3/users/9fce6b634ae74c4c91ffd8f7e2508613` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.828290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=9fce6b634ae74c4c91ffd8f7e2508613)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:40.829786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:40.831403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 165/324] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:40 2022] GET /identity/v3/users/9fce6b634ae74c4c91ffd8f7e2508613 => generated 294 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:40.837215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:40.860299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:40.868789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 160/325] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:40.890647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:40.940683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0067f77-0e43-4a72-b5da-62eadad30cb5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:40.974400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:40.975013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:40.975292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:40.975562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:40.976341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] RBAC: Adding query filter params (role.id=5a1e89be85c547438247b00c04c09046, user.id=9fce6b634ae74c4c91ffd8f7e2508613, scope.system=all) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:40.976675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:40.978092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:40.990945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 166/326] 10.222.0.144 () {56 vars in 1313 bytes} [Tue Jul 26 16:13:40 2022] GET /identity/v3/role_assignments?user.id=9fce6b634ae74c4c91ffd8f7e2508613&scope.system=all&role.id=5a1e89be85c547438247b00c04c09046 => generated 512 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:40.994211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:41.005358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd6f9c-2936-45c0-bcce-370cf2168b68 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.044292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.044899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.045179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.045445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.058417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8ea40488-c63d-4dc2-b714-724c0cdf44e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:41.063740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:41.064656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:41.065460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:41.066841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 161/327] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:41.071823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:41.109858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29a1f-498f-4aa1-be59-1653d44401d9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.110200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29a1f-498f-4aa1-be59-1653d44401d9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.110426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29a1f-498f-4aa1-be59-1653d44401d9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.111809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 162/328] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:41.117705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.118791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.118791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.141685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:41.144770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:41.163643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:41.190412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a39c76-6e5b-43b7-ada1-ba0a6bb1f687 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.219770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 163/329] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:41.227153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.227773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.228058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.228335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.229090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:41.229400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:41.230770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:41.238723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 167/330] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:41 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:41.242971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:41.244553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:41.321473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:41.339370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef290c-74db-4b74-b5b3-22b3154b1fe9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.340975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2225bb-d165-4cab-937a-fcb46c818ecf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.372936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.373578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.374045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.374457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.374612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.375259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.375541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.375837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] PATH_INFO: `/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.376015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:41.377009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=4e80d013c7c14d9e98c1a0565c90ba9b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:41.377484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:41.378442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:41.379053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 164/331] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:41 2022] GET /identity/v3/domains/default => generated 200 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:41.384477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 168/332] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:41 2022] GET /identity/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b => generated 294 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:41.384602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:41.385477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:41.396050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:41.481107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2a31-a391-4b49-884f-8d31ee95bbf0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.483399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831706-0468-49cc-9b00-c0d2503a4ca3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:41.516801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.517452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.517731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.518001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] PATH_INFO: `/v3/system/users/4e80d013c7c14d9e98c1a0565c90ba9b/roles/a6b6bec99f8947f2ac388c27d6e1fd89` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.518123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:41.518737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:41.519019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:41.519293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] PATH_INFO: `/v3/groups` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:41.519478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4e80d013c7c14d9e98c1a0565c90ba9b, role_id=a6b6bec99f8947f2ac388c27d6e1fd89)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:41.519970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:41.521052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd36d0-9da9-49df-bc6f-8045025ba27c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:41.521442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84174-8de2-4b0c-bf2b-452d898cdebc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:41.586362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 169/333] 10.222.0.144 () {58 vars in 1339 bytes} [Tue Jul 26 16:13:41 2022] PUT /identity/v3/system/users/4e80d013c7c14d9e98c1a0565c90ba9b/roles/a6b6bec99f8947f2ac388c27d6e1fd89 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:41.616068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 165/334] 10.222.0.144 () {60 vars in 1139 bytes} [Tue Jul 26 16:13:41 2022] POST /identity/v3/groups => generated 227 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:13:43.713246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0624be-5cdb-4cfc-a978-9a0b8bd4b4cf None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:43.714180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0624be-5cdb-4cfc-a978-9a0b8bd4b4cf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:43.714654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0624be-5cdb-4cfc-a978-9a0b8bd4b4cf None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:43.716437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 170/335] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:43.722195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:43.723075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:43.724170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:43.754641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:43.755232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:43.757705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:43.921165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58646b0-d2e3-4d13-81ee-0432af03a5a9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:43.921886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58646b0-d2e3-4d13-81ee-0432af03a5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:43.922386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58646b0-d2e3-4d13-81ee-0432af03a5a9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:43.924269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 171/336] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:43.925603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 166/337] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:43.945048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:43.945454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:43.945736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:43.970319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:43.974138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:43.975475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:44.088241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 172/338] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:44.103818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7baa9-4f7a-4baf-a23e-0fd8c407645d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.128737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.132970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:44.147793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.148468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.148751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.149023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.164668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:44.173978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:44.183752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:44.184686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:13:44.185718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:44.187446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 167/339] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:44.193399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.282436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676afc5-dbfb-4536-b64c-7cfbb3d03859 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.321334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.321985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.322268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.322542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.334457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:44.339765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:44.340689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:44.341464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 173/340] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:44.348539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.430379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87a6cb-0ae7-44cc-b265-eba39612ce62 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.475951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:44.511686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.512726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.513240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.513713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.514635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:44.515143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:44.516825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6972df-49b6-411b-ae1c-705fc0626bf6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:44.522537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dfbb2-b75a-4b70-b17b-b3b2fa53506e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.532033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 168/341] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:44.533612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ea49-b4f5-4368-81ba-5f21fda2c15d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.534107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ea49-b4f5-4368-81ba-5f21fda2c15d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.534606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ea49-b4f5-4368-81ba-5f21fda2c15d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.536337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 169/342] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:44.541291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.541795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.542269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.563529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.564151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.564458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.564796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.565524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:44.565884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:44.567842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:44.568053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:44.569494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:44.580118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 174/343] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:44.581035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.705164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb8a65-dc2b-4d7a-afbb-eb3a6edc6f50 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.714719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:44.731184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:44.736769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 170/344] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:44.737757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.744516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.745407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.745864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.746304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] PATH_INFO: `/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.747644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] RBAC: Authorizing `identity:get_user(user_id=4e80d013c7c14d9e98c1a0565c90ba9b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:44.749361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:44.752296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 175/345] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/users/4e80d013c7c14d9e98c1a0565c90ba9b => generated 294 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:44.763975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.885824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dd251-ab72-4cfc-ab8a-ab583ae0c457 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.924412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d636c-0255-4f87-87fc-d36dbfb6b4a0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:44.930492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.931458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.931982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.932491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:44.933457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] RBAC: Adding query filter params (role.id=a6b6bec99f8947f2ac388c27d6e1fd89, user.id=4e80d013c7c14d9e98c1a0565c90ba9b, scope.system=all) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:44.934064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:44.936049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:44.962505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 176/346] 10.222.0.144 () {56 vars in 1313 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/role_assignments?user.id=4e80d013c7c14d9e98c1a0565c90ba9b&scope.system=all&role.id=a6b6bec99f8947f2ac388c27d6e1fd89 => generated 512 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:44.963285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:44.974875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:44.975527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:44.975813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:44.976089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.066861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.125269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.152837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:45.154314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:45.178185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ffb5-6ee7-4078-b7e9-ed441228ee1c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.241879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.242456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.242740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.243019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.265481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-72f0447d-a0ce-4416-a1a8-ebc6c4a9a883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:45.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:45.272860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:45.273650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 177/347] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:45.274755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8b78-7b03-4ae6-9000-1f92be5c6be3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.274755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8b78-7b03-4ae6-9000-1f92be5c6be3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.274973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8b78-7b03-4ae6-9000-1f92be5c6be3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.276528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 178/348] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:45.281923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.282215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.282537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.292115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 171/349] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:45.293075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.317687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:45.318905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:45.410298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.427596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871b08-b133-4fdf-b6ed-149a503cb18b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.437074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 179/350] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:45.438219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.474634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.475383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.475637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.475913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.476671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:45.477760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:45.478452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:45.485317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.499227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 172/351] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:45.499227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.548593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884230ec-76a5-499c-b258-485a8ddcdc38 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.585140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.585899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.586289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.586632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.587853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:45.589480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:45.591588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 180/352] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:45.592507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.617301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce15b-86fb-4ef4-87d9-89b952f4ad3a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.656288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.657031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.657421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.657705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.671356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:45.677801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:45.678714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:45.679508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 173/353] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:45.680185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.705105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1561-437f-402f-9d83-8e8f09853d62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.705790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.718440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.746820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.747536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.748017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.748521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.827276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02de4b9-496e-4dce-9e7a-7dc4c3b206be None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:45.836060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:45.837724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbca4b-a75c-48cb-9e6c-7ab5338df815 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:45.871788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:45.872463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.872752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.873031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.879783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:45.882189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:45.883178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 174/354] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:45.883730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.955745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 181/355] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:45.957130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08718-f7e0-43f4-afd3-d24ca4916fb9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:45.957455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08718-f7e0-43f4-afd3-d24ca4916fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:45.958262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08718-f7e0-43f4-afd3-d24ca4916fb9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:45.959767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 182/356] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:45.960514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c08718-f7e0-43f4-afd3-d24ca4916fb9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:45.972212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:45.979299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.005868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0c51-f92d-471d-8015-73b39a26cdfb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.045186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.045810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.046883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.047373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.048188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:46.048798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.050626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57eda4-694a-442e-af1d-a8d51694922e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.061398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 175/357] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:46.063013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377919-8434-4e6b-99d4-f409b1b9c30c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.063306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377919-8434-4e6b-99d4-f409b1b9c30c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.063583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377919-8434-4e6b-99d4-f409b1b9c30c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.064897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 176/358] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:46.065921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.066173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.066445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.086049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:46.087568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:46.087695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c08718-f7e0-43f4-afd3-d24ca4916fb9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.127175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.127887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.128338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.128709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.130994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:46.130994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.131942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.138880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 183/359] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:45 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=5a1e89be85c547438247b00c04c09046 => generated 261 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:46.139691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.173113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 177/360] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:46.174357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.264995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.266799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.307531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8566263-9f92-41ad-93ed-0dd08b443a78 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.308221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b521ee-6735-4a2d-bb01-6dc4dc2b4d0d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.349050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.349716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.349997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.350637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.351441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.352157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.354362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.354362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.355391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.363101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc3bfe-3940-4909-a2b3-6b69134a7a70 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.363101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 184/361] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:46.363101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.363101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.363101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.381816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:46.383947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:46.437026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.438808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.466488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 185/362] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:46.467643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.529803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.558155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.570946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 178/363] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:46.572109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.598373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e6aad-f502-4b3b-b010-af8703738da2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.636710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.637091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.637254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.637588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.649430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:46.654449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:46.655375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:46.656148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 186/364] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:46.656855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.709002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895adc76-e164-442f-8699-632c6a9c5256 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.753276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.753905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.754710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.755002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.756187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.757656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.759427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 179/365] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:46.760368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.791992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee85000-c9ea-4d98-bb1a-b0f5928b78bc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.802178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.827576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:46.834931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.835860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.836144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.836448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.849671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:46.854954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:46.855870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:46.856663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 187/366] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:46.857813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.883479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92251-bfb2-4a1c-84d7-bf0703357bd2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:46.923335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:46.923906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:46.924186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:46.924491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:46.926603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:46.928066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:46.930100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 180/367] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:46.930872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:46.986413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21731e-15db-427c-9e33-f4649426fb6b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.030336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.031069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.031519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.031808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.032870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:47.033186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.035507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.047478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 188/368] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.048690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.050145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649fa7d-1776-4126-89b8-f84dc5d9da06 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.054486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.091401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.092263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.092654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.092933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.094294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:47.094294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.095880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.112479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 181/369] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:46 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 261 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.113511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.132102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.217812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad599b-61fc-40fe-bf2d-e8a923001f2c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.268422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.269179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.269557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.269881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.270689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:47.271001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.272570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdac05-d8a3-4b74-9800-c4e51e4a1939 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.272711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.284541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 189/370] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.286060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.329961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.330849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.331202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.331550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.333081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.334685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.336942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 182/371] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.341244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.384819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.421070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a182d6-1ef2-4213-be42-b934d0c19c37 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.441945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.466391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.467099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.467500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.467912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.476581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad28d3-6f81-499c-a6a9-3992e6a818b3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.523901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.524786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.525197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.525938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.572108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.572108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.626394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.627819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.667805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.719162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 190/372] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:47.720009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.736902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:47.768582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 183/373] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:47.771005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.833647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1963-8ee6-4b2c-96e0-507d75224068 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.881072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.882132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.882613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.883898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:47.884682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.887109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.895454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784524-6859-44f1-b2f9-d53c4bfeee29 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:47.896473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 191/374] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=5a1e89be85c547438247b00c04c09046 => generated 261 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.897800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.938965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:47.939731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:47.939930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:47.940208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] PATH_INFO: `/v3/projects/bf7f222516294286bf361e9d63865e13` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:47.941442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] RBAC: Authorizing `identity:get_project(project_id=bf7f222516294286bf361e9d63865e13)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:47.942928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:47.945329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 184/375] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/projects/bf7f222516294286bf361e9d63865e13 => generated 345 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:47.952308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:47.957171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.015171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.042095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1279-b87d-424a-997f-91d8534afbcd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.087304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.088126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.088532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.088909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.090201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:48.091746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:48.094516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 192/376] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:48.099008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:48.105970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101959cf-5348-481f-81cb-3c1486af38da None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.173572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.174483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.174907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.175319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] PATH_INFO: `/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.181568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] RBAC: Authorizing `identity:get_group(group_id=dc47fb8bddbe435cbf09eb6cddafc3eb)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:48.183203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:48.185603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 185/377] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:47 2022] GET /identity/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb => generated 230 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:48.194438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:48.238270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716316a-e8b3-4941-8e09-af6bb971d3c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.281382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.282601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.282989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.283367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.285054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:48.287434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:48.289956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 193/378] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:48 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:48.297530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:48.303280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.325468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.355637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857e5d-edac-444e-aaed-74c23d4db495 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.406900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.407513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.407789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.408057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.409028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] RBAC: Adding query filter params (group.id=dc47fb8bddbe435cbf09eb6cddafc3eb, role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=bf7f222516294286bf361e9d63865e13) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:48.409351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:48.413460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057cb95-afc3-4b82-adbf-57758d88ad13 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:48.419962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 186/379] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:48 2022] GET /identity/v3/role_assignments?group.id=dc47fb8bddbe435cbf09eb6cddafc3eb&scope.project.id=bf7f222516294286bf361e9d63865e13&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 261 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:48.421222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f03-5bb1-426f-9474-3fb3c55941f4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.421508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f03-5bb1-426f-9474-3fb3c55941f4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.421782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb5f03-5bb1-426f-9474-3fb3c55941f4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.430153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 187/380] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:48 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:48.435818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.436124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.436369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.448454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435f8c6-4a7c-4afe-ac1e-a6e1cd99ed4e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.467233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:48.481143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:48.498082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.498801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.499158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.499505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.500377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:48.501570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:48.503088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:48.521115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 194/381] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:48 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 261 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:48.598724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 188/382] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:48.625599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:48.733836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.765293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ee4d-cd53-4002-9b8c-6b33d60b7c21 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:48.813317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:48.813964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:48.814255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:48.814528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:48.829520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:48.835470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:48.837201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:48.838255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 195/383] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:48.845512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:48.936776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:48.992429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1fa38-89aa-49f7-b764-79ee000777b1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:49.047867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:49.048517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.048803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.049073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.049805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:49.050112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:49.051529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:49.060902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 189/384] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:48 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:49.069481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:49.242645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7267f8-16df-4be2-9cdb-92f7ed7e32ac None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:49.293071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:49.294073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.294558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.295048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.296472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:49.298129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:49.299955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 196/385] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:49 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:49.306319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:49.458125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b429307-67f7-4dcf-973a-0f8827afb36a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:49.493574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:49.499588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:49.500241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.500547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.500825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.505349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:49.506814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c8286-1b58-43e2-99db-408e8d48ef5f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:49.508793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 190/386] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:49 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:49.514172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:49.525687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:49.636750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ae223-d28d-4a23-bb32-6da9cd745773 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:49.671809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ed286-e13a-46b0-8569-92375ef6014d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.672459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ed286-e13a-46b0-8569-92375ef6014d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.673049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ed286-e13a-46b0-8569-92375ef6014d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.674440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:49.675141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.675365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 191/387] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:49.675460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.676499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/groups/e73577482b364ab38cb1a7749db4d3fc/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.681243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, group_id=e73577482b364ab38cb1a7749db4d3fc, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:49.682708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:49.686803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:49.687380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:49.687912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:49.713420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:49.714932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:49.796687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 197/388] 10.222.0.144 () {58 vars in 1447 bytes} [Tue Jul 26 16:13:49 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/groups/e73577482b364ab38cb1a7749db4d3fc/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:49.808032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 192/389] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:49.845942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:49.933972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:49.974773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3764c06-8f34-406e-a048-0a56364f407f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:50.016059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:50.016732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.017065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.017347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.028770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:50.034213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:50.035055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:50.035869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 198/390] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:50.042718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:50.178543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74edec7-4584-499d-acd5-b1b9edfe23aa None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:50.202931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:50.219975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:50.220624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.220908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.223427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.223427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:50.223427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:50.224067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:50.232939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 193/391] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:50 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:50.238252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:50.430667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea082c9-be05-4b27-aceb-70130108cc12 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:50.469296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:50.485666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:50.486314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.486599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.486873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.497425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:50.497425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb978d6-1403-46a8-832f-0dcec9f876e5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:50.497425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 199/392] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:50 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:50.498168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:50.525392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3d659-10ba-4f69-b475-6156cd0de8ad None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.525719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3d659-10ba-4f69-b475-6156cd0de8ad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.525940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3d659-10ba-4f69-b475-6156cd0de8ad None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.527560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 200/393] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:50.533599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.533878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.534155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.569410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:50.570608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:50.613968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13928-b411-4179-8ddf-31d34e35de06 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:50.648895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:50.649549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.649839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.650116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.651302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:50.652838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:50.665625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 194/394] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:50 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:50.665625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:50.759919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:50.772499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 201/395] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:50.819378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:50.852249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67cb1-9f1b-487e-b5da-f78af707e1fe None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:50.918350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:50.919034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:50.919572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:50.919887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/groups/e73577482b364ab38cb1a7749db4d3fc/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:50.923897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a14eac124b44ccaf102f87ccd1a07c, group_id=e73577482b364ab38cb1a7749db4d3fc, role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:50.925378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:50.971930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9a188-20cf-4dfd-abdf-cb0818c2f986 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.002739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 195/396] 10.222.0.144 () {58 vars in 1447 bytes} [Tue Jul 26 16:13:50 2022] PUT /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c/groups/e73577482b364ab38cb1a7749db4d3fc/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:13:51.011816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.026343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.027297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.027721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.028131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.049255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bb22b930-c6c6-4055-80da-d95d703713f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:51.063259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:51.064177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:51.064983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:51.067061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 202/397] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:51.072772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.086014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23945b-65e3-47d3-be51-eb94da48f884 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.086290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23945b-65e3-47d3-be51-eb94da48f884 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.086559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23945b-65e3-47d3-be51-eb94da48f884 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.088169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 203/398] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:51.093800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.094088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.094364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.136440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:51.137709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:51.215214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce489e1-641f-4338-a6d3-470010092c1d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.230634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.255292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.255933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.256224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.256538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.257343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:51.257667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.259137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.268761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 204/399] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:51.280481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 196/400] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:51.285997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.295059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.404837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172519ad-7894-41ba-9555-7e3d05857b1c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.441504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.442334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7aa74-dae1-4559-be27-f852317af347 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.442725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.443195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.443681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.454458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:51.460544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:51.461389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:51.462146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:51.464745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 197/401] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:51.473551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.477469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.494347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.495061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.495384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.495917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.496946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.498511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab300270-eb09-4b2f-9af1-19836a4c4352 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.503505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 205/402] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:51.505142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10623-3f37-44a6-92b0-ce31f63223fb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.505325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10623-3f37-44a6-92b0-ce31f63223fb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.505565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10623-3f37-44a6-92b0-ce31f63223fb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.509103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 206/403] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:51.510073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b10623-3f37-44a6-92b0-ce31f63223fb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.522317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.590361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae292ec-eafe-4f0f-bbb1-c5252b596d21 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.624503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b10623-3f37-44a6-92b0-ce31f63223fb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.626140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.626974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.627473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.627948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.628886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:51.629419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.630975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac31cb-9a8a-476c-aea9-cb1bd77ae418 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.640012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 198/404] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:51.646450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.646954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.647387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.664353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.665088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.665378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.665656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.666884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.667346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:51.668577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.668992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:51.676743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 207/405] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:51.677833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.737584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.740377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 199/406] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:51.741429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.768491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.798081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d468-17c2-4773-ba98-e172a6bb35a2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.834412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.835053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.835333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.835608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] PATH_INFO: `/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.837177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] RBAC: Authorizing `identity:get_group(group_id=dc47fb8bddbe435cbf09eb6cddafc3eb)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.838469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.844585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 208/407] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb => generated 230 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:51.844585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.854783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bd8f9-5900-4ac1-96d2-e0ce5f21956c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.890911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.891917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.892430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.892916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/groups/e73577482b364ab38cb1a7749db4d3fc/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.895938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, group_id=e73577482b364ab38cb1a7749db4d3fc, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:51.897618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:51.947161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464acd44-15aa-4a19-b788-4567dfdf6bc6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:51.965432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:51.978044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 200/408] 10.222.0.144 () {58 vars in 1447 bytes} [Tue Jul 26 16:13:51 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/groups/e73577482b364ab38cb1a7749db4d3fc/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:13:51.979712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:51.983010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:51.983688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:51.984044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:51.984514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:51.998237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:52.004574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:52.010517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:52.011313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 209/409] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:52.012193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.096597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:52.116379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26921-6370-4dd2-b153-00f5f7f09804 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.162892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.164217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.164730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.166649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] PATH_INFO: `/v3/projects/bf7f222516294286bf361e9d63865e13` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.169214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] RBAC: Authorizing `identity:get_project(project_id=bf7f222516294286bf361e9d63865e13)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.171679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.180824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdf2c1-b588-4b5e-850b-652ed012a41c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.180942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 201/410] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:51 2022] GET /identity/v3/projects/bf7f222516294286bf361e9d63865e13 => generated 345 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:52.180942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.222435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.222435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.222435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.222866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.223529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:52.223846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.226476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.239324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 210/411] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:52.244033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.301369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:52.308330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b558-c623-4739-9f4a-0eb690960d85 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.346703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.347292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.347575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.347881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] PATH_INFO: `/v3/projects/bf7f222516294286bf361e9d63865e13/groups/dc47fb8bddbe435cbf09eb6cddafc3eb/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.349885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf7f222516294286bf361e9d63865e13, group_id=dc47fb8bddbe435cbf09eb6cddafc3eb, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.351330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d40423-7336-429a-b176-e9d34f569c24 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.364895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbb6ec-9e56-4f5a-897b-84bf642c0998 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.400330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.401113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.401441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.401719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.402870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.404539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82667958-6d9b-4b82-84fa-6f6cc2b83f85 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.406307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 211/412] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:52 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:52.407502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc42-9334-4f36-9062-b902c8c36043 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.407849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc42-9334-4f36-9062-b902c8c36043 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.408208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc42-9334-4f36-9062-b902c8c36043 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.409724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 212/413] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:52.410903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 202/414] 10.222.0.144 () {58 vars in 1447 bytes} [Tue Jul 26 16:13:52 2022] PUT /identity/v3/projects/bf7f222516294286bf361e9d63865e13/groups/dc47fb8bddbe435cbf09eb6cddafc3eb/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:13:52.417683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bacc42-9334-4f36-9062-b902c8c36043 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.418700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.419189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.420080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.444750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:52.446402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:52.459030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:52.586066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 203/415] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:52.610396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bacc42-9334-4f36-9062-b902c8c36043 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.617688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.628761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:52.648996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.649765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.650229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.655407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.655407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.655407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.655407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 213/416] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:52 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:52.661223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.767548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a9e2e-555d-4452-8ed4-a15f4b4370b3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.770360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e593-909e-4b19-90aa-66e12eeceea0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:52.808134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.809100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.809484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.810128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/groups/e73577482b364ab38cb1a7749db4d3fc/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.812456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a2628bafaf24b09b35ca5c617abd8dc, group_id=e73577482b364ab38cb1a7749db4d3fc, role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:52.814220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:52.849255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:52.850104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:52.850489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:52.850866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:52.865574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:52.868787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:52.869648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:52.870581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 204/417] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:13:52.875966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:52.886188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:52.892904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 214/418] 10.222.0.144 () {58 vars in 1447 bytes} [Tue Jul 26 16:13:52 2022] PUT /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc/groups/e73577482b364ab38cb1a7749db4d3fc/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:13:53.017602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe98c-6724-41df-96d0-2369710b813f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:53.055273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:53.055900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.056182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.056483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.057291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:53.057601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:53.059071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:53.069239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 205/419] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:52 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:53.075134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:53.115375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:53.191571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ca9cf-a095-48de-a98f-8bfa507c6977 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:53.230027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:53.230658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.230938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.231212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.336317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:53.337807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:53.394473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:53.485102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 215/420] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:53 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:53.490642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:53.570970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:53.620041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9e3d-302f-4e66-8673-7c8e6729355c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:53.657010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:53.657647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.657929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.658202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.659362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:53.660931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6d390-ec84-4632-91cb-b998b64e9238 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:53.662930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 206/421] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:53 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:53.668421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:53.774225 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7227-715e-433e-ad44-692e00c7b1c5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:53.795836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff096f-1fc1-49c4-8c66-f7c0d227a299 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.796137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff096f-1fc1-49c4-8c66-f7c0d227a299 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.796400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff096f-1fc1-49c4-8c66-f7c0d227a299 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.797979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 207/422] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:53.803276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.803565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.803844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.828110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:53.829335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:53.830730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:53.831384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:53.831679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:53.831964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:53.834442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:53.835970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:53.837655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 216/423] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:53 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:53.848275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:53.860927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:53.942759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 208/424] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:13:53.967110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.049098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a42bb-e9ec-4e06-8366-b92ad24bb000 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.089267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.089890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.090158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.090434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.090937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.091233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:54.091515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:54.093126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:54.100215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 217/425] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:53 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=5a1e89be85c547438247b00c04c09046 => generated 613 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:54.198554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf5f4a-b1e5-45f1-9823-55ad2de6566d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.255003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.255661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.255940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.256208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.272897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-15ddf568-97d6-4424-a092-e096cfe7a132 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:54.277843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:54.278691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:54.280049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 209/426] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:54.284940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.362366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.367894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233bab-9679-44b9-9f5b-d05d2696aaf8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233bab-9679-44b9-9f5b-d05d2696aaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233bab-9679-44b9-9f5b-d05d2696aaf8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 210/427] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.401362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2abe69-d95d-4594-a62d-724f213461a2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.414120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:54.416037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:54.442150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.442779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.443061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.443338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.444056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:54.444843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:54.446232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:54.457985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 218/428] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:54 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:54.463708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.530099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 211/429] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:54.560293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.603328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf25a-b66e-4c71-a9c5-c8a267994822 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.635991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.642678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.643621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.644147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.645222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.647884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.700894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47528f5-6024-455d-a081-8d2f487aedbf None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.735637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:54.737232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46078ce1-2bb2-4230-a45e-7a2de45d9a71 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:54.742916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.743604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.743884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.744162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.755903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:54.764380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:54.765266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:54.766016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 212/430] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:54.774516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.862234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 219/431] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:13:54.866500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673bc5-bde8-4857-b9ea-226d04725431 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.866790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673bc5-bde8-4857-b9ea-226d04725431 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.867069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673bc5-bde8-4857-b9ea-226d04725431 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.869317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 220/432] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:54 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:54.871280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe673bc5-bde8-4857-b9ea-226d04725431 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:54.911492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.912490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac331c-8da4-4313-975a-b141abf8e4f4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:54.933034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:54.953122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:54.953750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.954037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.954371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.955098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:54.955409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:54.956813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee07b-6ebe-472a-af07-7a2fbb4b6e7a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:54.965977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 213/433] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:13:54 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:54.967295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:54.967633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:54.967911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:54.987434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:54.989200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:54.996965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe673bc5-bde8-4857-b9ea-226d04725431 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.033753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.034710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.035690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.035970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] PATH_INFO: `/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.037169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a14eac124b44ccaf102f87ccd1a07c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.038616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.040566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 221/434] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:54 2022] GET /identity/v3/projects/d4a14eac124b44ccaf102f87ccd1a07c => generated 299 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.041353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.069288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 214/435] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:55.071005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.145682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.150146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba408fb5-1a08-4ed7-a309-5f06f06c11b0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.184182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eaad1-bee8-4286-8082-7b35b3bfa6db None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.187082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.187698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.187979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.188252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.192297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.221167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.222090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.222367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.223405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.224652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.226662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.228422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 215/436] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.234168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.264368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.266173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.340217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769021cc-29fe-4995-886b-3ab570b97725 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.371455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.375700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.381081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.381218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 222/437] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:55.381308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.381308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.381479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.392461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:55.398127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:55.399178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:13:55.400124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:13:55.404222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 216/438] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:55.405107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.440008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.493286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbd5c-c384-4ef1-be1f-e105a96f7574 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.523530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae339f-75ae-48fe-b602-7d1ec72efbbb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.527949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.528658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.528940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.529335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.530121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:55.530429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.531848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.544440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 223/439] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=d4a14eac124b44ccaf102f87ccd1a07c&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 613 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.544440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.563839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.564543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.564825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.565100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.566939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.568442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0a0b-18b6-4c34-8232-7177a541035f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.570188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 217/440] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.576169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe83ac-1228-4f08-aeb0-88cb787950d5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.576496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe83ac-1228-4f08-aeb0-88cb787950d5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.576754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe83ac-1228-4f08-aeb0-88cb787950d5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.578305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 218/441] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:55 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:55.578853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe83ac-1228-4f08-aeb0-88cb787950d5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.625282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.670438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fe7f1-8eb5-424d-9481-2591451684fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.692548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe83ac-1228-4f08-aeb0-88cb787950d5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.694083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.712234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.712892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.716465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.716465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.716465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:55.716465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.716974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7eedb-4844-4d76-a171-16468bc1a9af None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.727835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 224/442] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.729293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.729584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.730478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.732743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.733391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.733671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.734001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.735164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.736627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.740075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 219/443] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.740776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.751124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:55.752316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:55.836170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 225/444] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:55.837154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:55.876213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7b5e-80b6-4051-b78e-da20c19b358f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.905959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.918515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.919142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.920115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.920115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.953388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd31dc-d379-49a3-a54f-fe1f815634f6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:55.958930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:55.986789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.987769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:55.988257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.988735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:55.989017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:55.989297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:55.990092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=5a1e89be85c547438247b00c04c09046, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:55.990407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:55.991816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:55.997553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 226/445] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=5a1e89be85c547438247b00c04c09046 => generated 613 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:55.998331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.109714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 220/446] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:13:56.112187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f7f9d-8a0e-4234-bbf0-2a283f415068 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.115303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.131912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.150609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.151260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.151546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.151825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.163727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:13:56.169905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:13:56.170960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:13:56.171712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 227/447] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:13:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:13:56.177431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.200016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.241198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b621-959f-4c41-bd0b-94ff3335d820 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.285885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.286539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.286816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.287092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] PATH_INFO: `/v3/projects/bf7f222516294286bf361e9d63865e13` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.288262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] RBAC: Authorizing `identity:get_project(project_id=bf7f222516294286bf361e9d63865e13)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.289738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.291715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 221/448] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/projects/bf7f222516294286bf361e9d63865e13 => generated 345 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:56.296239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0b6d9-6cf7-4669-9426-dc38f75c1f7f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.297290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.336985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.337667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.337882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.338157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.338873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:56.339177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.340595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.350080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 228/449] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:56.356761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.399377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.410423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3c2c-9596-4b72-9f5a-5af65a5cf426 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.444277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.444943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.445223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.445497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] PATH_INFO: `/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.446645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] RBAC: Authorizing `identity:get_group(group_id=dc47fb8bddbe435cbf09eb6cddafc3eb)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.448097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.449707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 222/450] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/groups/dc47fb8bddbe435cbf09eb6cddafc3eb => generated 230 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:56.455398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.457785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe94a3-dfd7-4b3d-9e69-55ec3e065003 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.464281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.491641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.492189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.493052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.493052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.558930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b105b-e4ad-4496-85b4-32ac46fcb0be None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.560211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.561718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.592701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.593307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.593586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.593857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.594639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] RBAC: Adding query filter params (group.id=dc47fb8bddbe435cbf09eb6cddafc3eb, role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=bf7f222516294286bf361e9d63865e13) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:56.594946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.596367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.601838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.602110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 223/451] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/role_assignments?group.id=dc47fb8bddbe435cbf09eb6cddafc3eb&scope.project.id=bf7f222516294286bf361e9d63865e13&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 613 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:13:56.663693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 229/452] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:13:56.669542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.742100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:56.772751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc313-4c01-44a1-b5cb-0363760ff4a1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.806577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.807236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.807507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.807829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] PATH_INFO: `/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.808977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] RBAC: Authorizing `identity:get_project(project_id=2a2628bafaf24b09b35ca5c617abd8dc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.810646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.812282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 224/453] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/projects/2a2628bafaf24b09b35ca5c617abd8dc => generated 303 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:56.818621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:56.912849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe428657-e57b-426e-8ea0-32c57104d610 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:56.944693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:56.945310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:56.945590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:56.945863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] PATH_INFO: `/v3/groups/e73577482b364ab38cb1a7749db4d3fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:56.946988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] RBAC: Authorizing `identity:get_group(group_id=e73577482b364ab38cb1a7749db4d3fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:56.948459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c165f2-3ff8-450b-9324-bfa8138f97ca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:56.950010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 230/454] 10.222.0.144 () {56 vars in 1185 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/groups/e73577482b364ab38cb1a7749db4d3fc => generated 227 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:56.955725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:57.052129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329899f-c238-48b8-b157-493dbbe3a219 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:57.084911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:13:57.085499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:57.085775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:57.086043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:57.086826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] RBAC: Adding query filter params (group.id=e73577482b364ab38cb1a7749db4d3fc, role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:13:57.087270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:13:57.088636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb28da8-01e1-4f1d-acc1-6a58a0c03ddd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:13:57.094164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 225/455] 10.222.0.144 () {56 vars in 1381 bytes} [Tue Jul 26 16:13:56 2022] GET /identity/v3/role_assignments?group.id=e73577482b364ab38cb1a7749db4d3fc&scope.project.id=2a2628bafaf24b09b35ca5c617abd8dc&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 613 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:13:59.545134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff709-1c46-4e81-99b3-ef6df463a502 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:59.546028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff709-1c46-4e81-99b3-ef6df463a502 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:59.546460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ff709-1c46-4e81-99b3-ef6df463a502 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:59.548215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 231/456] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:59.561407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288cdfd-586d-4945-807d-124c2c1dbc5c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:59.561886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288cdfd-586d-4945-807d-124c2c1dbc5c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:59.561955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288cdfd-586d-4945-807d-124c2c1dbc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:59.570581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:59.589312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a288cdfd-586d-4945-807d-124c2c1dbc5c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:59.590507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a288cdfd-586d-4945-807d-124c2c1dbc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:13:59.722468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 226/457] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:13:59.768468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff709-1c46-4e81-99b3-ef6df463a502 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:13:59.786571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:13:59.874001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccdbd5-0265-4727-9afa-11734226a070 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:59.874373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccdbd5-0265-4727-9afa-11734226a070 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:59.874587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccdbd5-0265-4727-9afa-11734226a070 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:59.876417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 227/458] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:13:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:13:59.888694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1039bc-186a-4091-8b2e-3b54117b9db8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:13:59.888979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1039bc-186a-4091-8b2e-3b54117b9db8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:13:59.889244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1039bc-186a-4091-8b2e-3b54117b9db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:13:59.924335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ff709-1c46-4e81-99b3-ef6df463a502 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:13:59.944912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-da1039bc-186a-4091-8b2e-3b54117b9db8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:13:59.946185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-da1039bc-186a-4091-8b2e-3b54117b9db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:00.010910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.011896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.012455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.015978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.047681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.051122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:00.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:00.081075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:00.082270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:00.082270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:00.082270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.082270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:00.084015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 232/459] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:00.092901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.132834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 228/460] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:13:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:00.138772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c572aa-402d-4a96-9bef-c791f50fbd37 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.141073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c572aa-402d-4a96-9bef-c791f50fbd37 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.142704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c572aa-402d-4a96-9bef-c791f50fbd37 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.148956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 229/461] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:00 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:00.153864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.154435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.154915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.178882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.185216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:00.186631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:00.223074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6ae31-6ee4-4396-b42e-f310d62efae4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.254862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.262209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.262850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.263156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.263436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.264129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:00.264634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.265839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.271169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 230/462] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:00.272866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.288203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 233/463] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:00.294681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.412475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1863e10-fe66-4390-8307-3ab6f501355e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.441613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b73cd-174d-4e82-8a4e-07ebb19b367b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.456162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.457204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.457666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.458016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.467903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.474883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.476892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.490224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.491217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.491690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.492152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.493397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.495041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b1313-3c03-404e-b513-39be60f87e7f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:00.499547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:00.500360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:00.501206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:00.501206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:00.501206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.501206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:00.504783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 231/464] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:00.505669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.551490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.596923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 234/465] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:00 2022] POST /identity/v3/users => generated 247 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:14:00.599529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922eb2-174f-4fac-805c-60a117cd18ff None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.599816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922eb2-174f-4fac-805c-60a117cd18ff None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.600093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922eb2-174f-4fac-805c-60a117cd18ff None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.601807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 235/466] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:00.606787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922eb2-174f-4fac-805c-60a117cd18ff None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.624777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbbec-afc7-4ed1-8d96-2347c862de68 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.679798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.681086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.682036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.682514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.699258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.701790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4de4828f-c69c-4c9b-ba92-fbf7fbd1672f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:00.723946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:00.724868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:00.725706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:00.725706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:00.725706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:00.725706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:00.725706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 232/467] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:00.726937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.727222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.727736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.739340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922eb2-174f-4fac-805c-60a117cd18ff None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.746861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:00.748104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:00.782057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.782682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.783011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.783285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.783972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:00.784277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.786967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.801053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 236/468] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:00.801900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.835353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 233/469] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:00.836202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.844519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.868800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:00.919833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d36ec-bd29-4d97-98dc-8a7db8af006d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.944993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58223db-51cc-4374-89bf-42003306f3d8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:00.957772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.958402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.958662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.958934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.959621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:00.962210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.962210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:00.973021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 237/470] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:00.973974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:00.994518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:00.995198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:00.995481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:00.995754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:00.996564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:00.998005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.074477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:01.080563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de846e4-3dcb-43a4-be14-42fc028f91ce None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:01.105093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 234/471] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:00 2022] POST /identity/v3/users => generated 249 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:01.105900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:01.149296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:01.149960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.150241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.150514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.166661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:01.168151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:01.202032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:01.202908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:01.203697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:01.203697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:01.203697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:01.203697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:01.203697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 238/472] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:01.209166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:01.263707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:01.302416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eddcf-30ba-4653-a18f-6cc5d722eae7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:01.353656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:01.354284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.354567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.354842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.357137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:01.358804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54b5f2-0752-4d33-991c-dddf1e01a960 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.379647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b944b-ad8a-4247-b961-010f5be71c6f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:01.414741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:01.415538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.415810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.416078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.416800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:01.417101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:01.418466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519d5e6-5c7c-4def-9304-98b047e52a25 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.430193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 239/473] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:01.431558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36cda-a8e7-4eb1-83e7-0ae0c85bdd3d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.431828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36cda-a8e7-4eb1-83e7-0ae0c85bdd3d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.432102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa36cda-a8e7-4eb1-83e7-0ae0c85bdd3d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.433507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 240/474] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:01.436290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa36cda-a8e7-4eb1-83e7-0ae0c85bdd3d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:01.448625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:01.474222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 235/475] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:01 2022] POST /identity/v3/users => generated 249 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:01.474222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.474222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.474222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.494053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:01.495894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:01.579556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 236/476] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:01.603763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:01.644783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa36cda-a8e7-4eb1-83e7-0ae0c85bdd3d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:01.709818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:01.710445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.710729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.711180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.711931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:01.714003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.716742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:01.761828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df064eae-b676-46e7-b3cd-88789c727db4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:01.798589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:01.799219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:01.799549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:01.799825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:01.801495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:01.803157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:01.818130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 241/477] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:01 2022] POST /identity/v3/users => generated 250 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:01.852742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.sql.core [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:14:01.852742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 16:14:01.852742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [parameters: {'id': '15e8df9e02e9444a9412c30f14c5e0b6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 16:14:01.852742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80369) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:01.873348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:01.874277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application yield resource Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise self.value Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application session.commit() Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:14:01.875149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:14:01.875969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:01.877682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:01.878461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [parameters: {'id': '15e8df9e02e9444a9412c30f14c5e0b6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:01.879211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 16:14:01.879959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:01.881935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 237/478] 10.222.0.144 () {60 vars in 1136 bytes} [Tue Jul 26 16:14:01 2022] POST /identity/v3/roles => generated 144 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:01.888695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:01.983113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:02.034248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25ae6-5b42-4270-b42e-d31292959a46 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:02.074619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:02.075269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:02.075553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:02.075830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:02.091937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c3295cd3-2bf4-4d13-9128-efc89110abfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:02.098040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:02.098876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:02.099640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 242/479] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:02.106074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:02.214165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc22b-d58c-41f1-8145-4d329306fe48 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:02.246393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:02.253527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:02.254151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:02.254431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:02.254703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:02.255424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:02.255732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:02.257163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d84abb-47c6-4210-9baf-163ebd3fa519 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:02.265989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 238/480] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:02.819847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a53c-3879-4098-89e4-b29965e17886 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:02.820333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a53c-3879-4098-89e4-b29965e17886 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:02.820333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a53c-3879-4098-89e4-b29965e17886 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:02.821967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 243/481] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:02.827177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:02.827578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:02.827760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:02.846787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:02.848576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:02.941152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 239/482] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:02.978842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85a53c-3879-4098-89e4-b29965e17886 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:03.166022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85a53c-3879-4098-89e4-b29965e17886 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:03.196930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:03.210505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:03.211144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.211421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.211693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.225361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:03.230327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:03.231178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:03.232027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 244/483] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:03.237386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:03.410835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2266517-7997-4707-90c9-d2dcf835b02f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:03.447933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:03.465072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:03.465704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.465995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.466277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.467014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:03.467317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:03.468726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048be5c-df6a-4d17-91e9-138307a7cdfb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:03.477993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 240/484] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:03.483444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:03.688272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b550-9a12-4f58-8781-36eed26dd2af None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:03.735588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee5da-1f85-42d2-82be-b134ce7e37f2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.735899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee5da-1f85-42d2-82be-b134ce7e37f2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.736065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aee5da-1f85-42d2-82be-b134ce7e37f2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.737728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 241/485] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:03.742742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.743031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.743315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.747822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:03.748791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.749307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.749911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.763268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:03.764491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:03.768711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:03.770989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:03.792939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:03.793811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:03.794643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:03.794643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:03.794643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:03.794643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:03.799808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 245/486] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:03.806121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:03.848862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:03.902741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 242/487] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:14:03.946644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3cd50-f8f3-45ca-a21d-d0d8990454ba None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:03.962031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:03.990941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:03.991504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:03.991786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:03.992062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:03.992814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:03.993125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:03.994555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.028702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 246/488] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:04.031222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.174377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce34ce-fb9d-466d-921f-374246ebec6c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.199993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.206453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc8ea5-c636-4d84-b720-bdf66336a0a8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.216122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.245133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.246726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.247722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.248341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.256208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.256908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.257182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.257458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.260873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.262540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.268356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-31a74a0b-e133-4569-81b5-71447b74deed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:04.273437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:04.274292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:04.275063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 243/489] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:04.280750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e2fb8-e9aa-4f3f-83ac-b3e6fb337d71 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.280750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e2fb8-e9aa-4f3f-83ac-b3e6fb337d71 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.280750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e2fb8-e9aa-4f3f-83ac-b3e6fb337d71 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.280750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 244/490] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e69d9048-1fb2-42a4-a28a-e429071ec215 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:04.284890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:04.285765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:04.286629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:04.287113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e2fb8-e9aa-4f3f-83ac-b3e6fb337d71 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.288883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 247/491] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:04.290215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.290740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.291182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.314915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:04.317201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:04.396413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 248/492] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:04.396413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.401897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e2fb8-e9aa-4f3f-83ac-b3e6fb337d71 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.438320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.438940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.439225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.439505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.440221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:04.440571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.441998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced303df-8e26-4da9-a2a7-6a58086158b1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.450004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 245/493] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:04.455651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2e24-ce4e-4b89-b18a-f72ef4494ca9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.455944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2e24-ce4e-4b89-b18a-f72ef4494ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.456223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2e24-ce4e-4b89-b18a-f72ef4494ca9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.462376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 246/494] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:04.462376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d2e24-ce4e-4b89-b18a-f72ef4494ca9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.502402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.508829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.518210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7a1d-ed8a-4b67-9edd-218edc687d4a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.553738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.554370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.554701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.554976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.555785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:04.556092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.557559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.578517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d2e24-ce4e-4b89-b18a-f72ef4494ca9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.600917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 249/495] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:04.601771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.636556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.638523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.639207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.639846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.656606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-213b962a-dc1a-495a-9790-d2595e923df0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:04.662061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:04.662958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:04.663731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 247/496] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:04.665001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.665001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.665154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.684965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:04.686145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:04.715746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ca3eb-e500-44dd-88e3-52946641ae90 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.750697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.751332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.751610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.751885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.764481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.764481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.767068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.768673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 248/497] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:04.770043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:04.780633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:04.783429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 250/498] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:04.784248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.803503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:04.881001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad07c6-e466-4cdc-89e7-b07caeef1762 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.899277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fd587-358d-4623-a68d-595e58ff1093 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:04.918331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.918955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.919239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.919596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.935456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:04.936069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:04.936351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:04.936656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:04.937362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:04.937666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.939039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:04.946863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 251/499] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:04.947658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:04.994472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:04.995929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.009672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.022687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.051712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d949b-3c99-4efb-b4ba-1b2ef0da170e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.084346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.085018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.085303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.085578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.093470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 249/500] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:05.094561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.095462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5a24c044-1486-48f9-84e0-ac3f4539ee0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:05.101713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.102678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:05.103440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 252/501] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.104069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58baa-c376-4aff-bef0-f83ed3ea16e3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.104269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58baa-c376-4aff-bef0-f83ed3ea16e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.104570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58baa-c376-4aff-bef0-f83ed3ea16e3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.105806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 253/502] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:05 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:05.106497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd58baa-c376-4aff-bef0-f83ed3ea16e3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.195838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15164d-f1b4-4a2b-bcc9-5457eb97b603 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.207408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd58baa-c376-4aff-bef0-f83ed3ea16e3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.221834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.225399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.230112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.230726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.233657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.233657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.233657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:05.233657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.233657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.241413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.242008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.242286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.242556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.245006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 250/503] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:05.245884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.258779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.260245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:05.274883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.275726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.276860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:05.276860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:05.276860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.276860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:05.276860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 254/504] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.277170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.350686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f2ef-897a-4899-82f2-b78547c0c7ad None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.376021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9cf8-96dd-441d-bdb2-3624e06e6be5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.383885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.384561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.384847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.385472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.396644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.398104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.410798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.411401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.411678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.411948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1fd6cd06-0e8a-408c-b02e-69efae6f571a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:05.412581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.413532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.414352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:05.414352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:05.414352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.414352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:05.414551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:05.414551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.414680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 251/505] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.414746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.415011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.415127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.415390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.422849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 255/506] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:05.423578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.433949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:05.435443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:05.455115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.469668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.503519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 252/507] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:05.504359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.525572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d31ff-6c7e-47f1-b4a2-8379bb39403e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.560202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.560869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.561150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.561424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.569085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.570526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:05.581589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.582453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:05.583316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 256/508] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.583887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.613669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc231774-d1bf-40ef-abb5-ef346cd7ca82 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.647863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.648509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.648794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.649154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.649766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:05.650074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.651442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.662899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 253/509] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:05.663685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.671759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.680445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079722-5ea3-4a5b-9229-7eeb5966fdb4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.696472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.712933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.713537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.713813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.714081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.714763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:05.715064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.716507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.728940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 257/510] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:05.729719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.765411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8748990-86d9-4383-beb8-58ee00be8d15 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.801067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.801624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.801901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.802171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.812789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.814210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:05.828206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.829254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.830076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:05.830076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:05.830076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.830076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:05.830076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 254/511] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.830348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035c45a-47b5-49d7-bed4-99b02cae53e3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.830418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.867812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.868478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.868764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.869038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.878802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.880243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:05.897119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:05.897975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:05.898801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:05.898801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:05.898801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:05.898801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:05.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 258/512] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:05.900604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:05.907821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.917111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:05.934201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c0d3f-c38e-4237-a27a-b45ef417cb4e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:05.966803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:05.967414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:05.967697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:05.967979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:05.968806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:05.969121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:05.970539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.001734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7389-6cab-4a4f-9ee1-46c33db9effc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.007265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 255/513] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:06.008087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.034823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.035452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.035731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.036002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.043409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.044882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:06.055863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:06.056779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:06.057628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 259/514] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:06.058163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.105483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4f58-5a16-4946-922c-89c809c96a3c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.117992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.138710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.139348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.139632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.139908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.146983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.148443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.150960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.155432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6308c-019e-41af-a413-9844a71ef824 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:06.160713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:06.161502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:06.162297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 256/515] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:06.163183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.193230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.193838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.194118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.194398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.195075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:06.195375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.196760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.208547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 260/516] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:06.209323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.262108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c649b3f-ab99-440b-b1e2-a599dd4c4d1e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.299596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.300230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.300540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.300811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.301496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:06.301799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.303162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.316711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 257/517] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:06.317450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.319057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d663-e92b-43fa-9ca2-37fb2add6611 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.343411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.353729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.354334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.354611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.358776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.376821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.418181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325af91d-4ae6-40f0-9af3-40a93045330c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.422362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.423841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.452262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.452893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.453168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.453438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.454222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:06.454521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.455994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.475828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 258/518] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:06.476693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.520375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 261/519] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:06.521239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.544931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.579720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732613-d6bf-424b-8c17-93c2e439e522 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.601283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.618637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.619266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.619547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.619887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a41160-637d-4553-96c7-129df42c07f2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.620156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.620947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:06.621352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.625346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.645096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 259/520] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:06.645915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.661773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.662413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.662739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.663013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.670070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.671510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:06.682918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:06.689666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 262/521] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:06.690701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.743356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597a23c-43b0-4e7c-b09f-93e46476be4a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.767237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.776687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.777281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.777558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.781858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.781858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.781858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.782679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9285cd0-9d70-47d1-a0e6-446524053bc2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:06.815036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:06.816199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:06.816550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:06.816828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:06.881145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:06.882597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:06.898331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 260/522] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:06 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:06.899149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:06.940197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.977091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:06.998823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 263/523] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:06.998823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.030601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c1148-5e94-4f5a-92f8-8ea8ef32e1c7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.075673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.076331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.076676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.076950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.088019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.089599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:07.105948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:07.106865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:07.107698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:07.107698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:07.107698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:07.107698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:07.107698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 261/524] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:07.108484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.114990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5da9e-8063-4ba7-9673-f80838bc0d1a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.152996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.153841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.154127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.154564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.155359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:07.155671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.157127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.162577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 264/525] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:06 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:07.163360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.215646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.232606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a368e36-3ac4-42bd-9391-7527d5cb9ed3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.240103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.273543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.273759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.273959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.274209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.275200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:07.275708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.277299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.297780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 262/526] 10.222.0.144 () {56 vars in 1144 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:07.298732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.311410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f41199-eeea-4f83-8ebe-8768aa2fe081 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.355701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.356724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.357317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.357784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.370730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.372792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:07.388818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:07.389706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:07.390512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:07.390512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:07.390512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:07.390512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:07.392371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 265/527] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:07.393586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.419735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6affa91-b8d2-452b-829d-7e3fb3751043 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.457580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.458206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.458481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.458756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.459450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:07.459749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.461253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.477064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 263/528] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:07.481854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.488281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.516246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.521526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca0e1-4495-474f-a16c-a7ea83e68a6d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.563987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.565492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.566450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.567133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.605776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e113c7-130a-4089-96e0-00c3b7475171 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.647258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.647886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.648166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.648470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.649180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:07.649603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.652727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.652851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.654618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.669304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 264/529] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:07.670052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.737651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.781273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:07.784468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef7217-bc8c-4e7c-b6b5-695fdd7d1e5a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.785165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 266/530] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:07.786530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.829732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.830649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.831075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.832379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.841298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.843339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:07.859162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:07.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:07.861187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:07.869878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 265/531] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:07.869878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.895825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca60625-b337-464b-92f9-eb16d43d15e0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:07.932345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:07.932695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:07.932942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:07.933216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:07.941335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:07.942831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:07.955264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:07.956059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:07.957519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 267/532] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:07.957996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:07.996643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7153fb6-072a-4588-a3d0-684b4dc56dc2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.025070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.032761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.035822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.039382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.039382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.039382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.049695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.052017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:08.071087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:08.072021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:08.072856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:08.072856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:08.072856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:08.072856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:08.075275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 266/533] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:08.076497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.082126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668535-d470-4c41-90e7-4f740fb7f77b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.126621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.127345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.127697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.128763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.129634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.130302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.131993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.155122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 268/534] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.156326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.197832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d994e-3a1f-427f-b5b5-f0cce66e8428 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.240660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.241753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.242145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.242523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.243706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.244259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.246116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.262020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 267/535] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.263213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.267110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6f66-a6ec-40c8-bd7c-40424f447885 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.300896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.305855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.306827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.307365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.307863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.308840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.309639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.311263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.320923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.325083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 269/536] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.328560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.396010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901277-4e78-4251-88e0-35042ba6103f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.437921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.439177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.439535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.439924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.441028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=b6d652014d6446a0ae162120aac0ed94) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.441414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.443562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.448637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176c6c-b04b-4ebc-9a74-ee207ac2fd6c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.450006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 268/537] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/role_assignments?user.id=b6d652014d6446a0ae162120aac0ed94&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.452970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.511422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.513087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.515038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.515533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.518401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=5361a7650f164ff8b11abfe6b3c4c382) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.519125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.522207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.543604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 270/538] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/role_assignments?user.id=5361a7650f164ff8b11abfe6b3c4c382&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.617280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe38e5-1ad9-46d7-92f2-fcba972db657 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.620844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.660104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.660963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.661351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.661790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.670386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.672051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:08.685271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:08.686447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:08.687317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:08.688464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 269/539] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:08.695046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:08.755209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:08.812963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137888-ab3c-4e72-9e7a-5b874b60f34e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:08.858579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:08.859208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:08.859494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:08.859776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:08.860575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:08.860889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:08.862320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feae9f0-f954-460a-8ead-1c7b1ae03898 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:08.875277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 271/540] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:08.881220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.009065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531fa72-2030-4d60-8637-c2fd79265088 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:09.045287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:09.045907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.046189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.046463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.047260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=bd7e03b6ec884bf2bf5039aefba4e5e4) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:09.047571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:09.049050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28cfdc-6c93-4de8-b602-975e40a571c1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:09.055118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 270/541] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:08 2022] GET /identity/v3/role_assignments?user.id=bd7e03b6ec884bf2bf5039aefba4e5e4&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:09.099461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447453-e02b-4581-afa4-5734f0e74695 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.099971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447453-e02b-4581-afa4-5734f0e74695 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.100045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447453-e02b-4581-afa4-5734f0e74695 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.101679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 272/542] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:09.107198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525997f-0fd5-4ca9-98ad-d78db84c4881 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.107740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525997f-0fd5-4ca9-98ad-d78db84c4881 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.108215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525997f-0fd5-4ca9-98ad-d78db84c4881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.134810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4525997f-0fd5-4ca9-98ad-d78db84c4881 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:09.136055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4525997f-0fd5-4ca9-98ad-d78db84c4881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:09.161398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:09.227918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 271/543] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:09.270148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5447453-e02b-4581-afa4-5734f0e74695 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.382632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdfe83-c136-4d7a-a6bf-afc28ee879b2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.382921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdfe83-c136-4d7a-a6bf-afc28ee879b2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.383200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdfe83-c136-4d7a-a6bf-afc28ee879b2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.384874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 272/544] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:09.390182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.390479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.390754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.405474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5447453-e02b-4581-afa4-5734f0e74695 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:09.410762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:09.411374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:09.412850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:09.487261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:09.487973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.491164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.491164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.517759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:09.524351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:09.525200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:09.525966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:09.532658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 273/545] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:09.538160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.545219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 273/546] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:14:09.608611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.656402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069af50-46b2-406d-a1e4-6fa524dd84f8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:09.692229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:09.693191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.693666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.694150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.695048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:09.695566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:09.697168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:09.706869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 274/547] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:09.712762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.818600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:09.843610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:09.851684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e129a-d1ad-4ae3-b4f8-5f33839f544c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:09.923728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:09.925194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:09.925561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:09.925850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:09.949763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:09.960574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce260d-7ecd-4889-93c7-3952cea28942 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:09.963661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:09.964637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:09.965375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 274/548] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:09.973590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:09.998610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:09.999698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.000193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.000713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.019552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.021424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:10.052543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:10.053429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:10.054247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:10.054247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:10.054247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:10.054247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:10.054247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 275/549] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:10.066317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.135680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d38eea-fb64-404f-a9f2-c88d7655fe13 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.205143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.206160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.206642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.207113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.208046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:10.208596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.210154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.223760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 275/550] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:10.228608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.230761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.247707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.274360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff00a1b-ae57-42c4-a9a8-fc41c0858e24 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.341119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.341752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.342035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.342310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.343011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:10.343317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.344722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.355762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfcd60-89ad-48ad-8f2e-891a39db9ebb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.370883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 276/551] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:10.377550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.446616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.447185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.447465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.447739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.464551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.466504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:10.499838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:10.500711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:10.501525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:10.501525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:10.501525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:10.501525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:10.502585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 276/552] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:10.503463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.567130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b7b1-ef05-4c1c-8ee1-82d0ce7a26ef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.624733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.627532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.628414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.628522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.628779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.642727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.649388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.660750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:10.669771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:10.670673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:10.671538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:10.673437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 277/553] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:10.678275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.703146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca5bcb-888d-4e84-b9e7-4b8bbdcf870d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.785768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.786806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.787288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.787744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.789040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:10.789620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.790952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f944e1-425f-42b9-831f-16e0db769c60 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.809923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 277/554] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:10.811212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5019e-1125-4e77-b958-0a4d4244dd17 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.815835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5019e-1125-4e77-b958-0a4d4244dd17 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.816154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5019e-1125-4e77-b958-0a4d4244dd17 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.817489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 278/555] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:10 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:10.818205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5019e-1125-4e77-b958-0a4d4244dd17 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:10.822202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf77a2-83d7-4172-bb13-ff8e22cfdbb4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.858277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.858901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.859186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.859522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.860554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:10.861724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.863779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e4b3-5b44-4ed4-b36b-711c344a4f33 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.886333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 278/556] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:10.888316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.888673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.888959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.913171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:10.914644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:10.933475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5019e-1125-4e77-b958-0a4d4244dd17 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:10.953140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.981029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:10.981029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:10.981029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:10.981029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:10.981781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:10.982827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:10.984698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:10.997279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:10.998948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:11.000095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:11.008736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 279/557] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:11.008736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.010197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 279/558] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:11.011258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.119392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6597cae-2756-4dbf-9c0c-230395926124 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.136900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3a373-a8f2-4cc3-8923-cdd462f754a3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.153007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.153624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.153902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.154171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.174016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.174631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.174941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.175213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.176015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:11.176321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.177782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.195860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 280/559] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:11.197257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.214345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.228443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.230148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb69e0-380e-4935-bc3c-ad744b979ce6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.236484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.302597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a283d-d901-406c-9557-3f44778e358f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.335610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 280/560] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:11.337558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945af2-2079-4696-a8c7-2b4dc5ed26d0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.337992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945af2-2079-4696-a8c7-2b4dc5ed26d0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.338459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945af2-2079-4696-a8c7-2b4dc5ed26d0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.339951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.340557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 281/561] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:11.340930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.341380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.341630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90945af2-2079-4696-a8c7-2b4dc5ed26d0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.341940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.354985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:11.359913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:11.361047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:11.362061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:11.365737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 281/562] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:11.366565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.431509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.447292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90945af2-2079-4696-a8c7-2b4dc5ed26d0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.460350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.466371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b2e01-563b-4668-8604-2d6dfb6e09a5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.485138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.485782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.486071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.486347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.500057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.500977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.501537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.501642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.501883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.502156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-69428858-0a9f-4c97-9136-490b051dcd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:11.517405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:11.518563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:11.520432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:11.520432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:11.520432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:11.520432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:11.527102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.527190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 282/563] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:11.527190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4c8cd-6169-43eb-b8b6-1007e06746ec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.527190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4c8cd-6169-43eb-b8b6-1007e06746ec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.527190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4c8cd-6169-43eb-b8b6-1007e06746ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.527437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.538546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0bd4c8cd-6169-43eb-b8b6-1007e06746ec None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:11.539357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:11.540240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:11.541869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:11.541869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:11.541869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:11.541869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:11.542366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0bd4c8cd-6169-43eb-b8b6-1007e06746ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:11.542516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 282/564] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:11.545371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.605018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 283/565] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:11.606231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf6e42-de31-4dc6-9be8-5a827c9e72e3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.606513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf6e42-de31-4dc6-9be8-5a827c9e72e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.606782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf6e42-de31-4dc6-9be8-5a827c9e72e3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.612039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 284/566] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:11.612039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf6e42-de31-4dc6-9be8-5a827c9e72e3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.648000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f3707-d98c-4b03-ba56-593df5875807 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.677605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.681565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.682182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.682458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.682727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.683428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:11.683729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.685448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.694065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 283/567] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:11.694751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.708105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.713034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf6e42-de31-4dc6-9be8-5a827c9e72e3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.747430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.748022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.748305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.748602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.749287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:11.749595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.750961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540af9-b440-48e7-b420-38b3780bbe60 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.762428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 285/568] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:11.763583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.763859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.764134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.782388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:11.783598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:11.794561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed592-ab88-4733-bb71-e7482104d176 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.832205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.832848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.833121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.833391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.834078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:11.834380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:11.835734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:11.847363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 284/569] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:11.848180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.856858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 286/570] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:11.857681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:11.914407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.916087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:11.946272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5840fb8-2f11-4203-8e40-4a296dd6aa21 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.955981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8f510-521f-48b8-a81c-5741f0fbfbef None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:11.980567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.981208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.981486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.981760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.990162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:11.990798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:11.991078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:11.991349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:11.992466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:11.998438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:11.999441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:12.000245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 285/571] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.001317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.001389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.002667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:12.017100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.018114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.019167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:12.019167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:12.019167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:12.019167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:12.019167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 287/572] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.019838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.103343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187b28-5fc7-4b77-92c4-12cd31bfbd12 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.126505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba67c1-c4ae-450f-b24a-45cf7c3ebe1b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.137156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.139381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.145881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.146523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.146807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.147077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.153613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.155051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.160052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.160702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.160986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.161262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:12.166247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.167097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:12.167939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 286/573] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.168437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.168511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.169151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:12.181025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.182106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:12.182969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 288/574] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.183432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.274789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef408-3a18-474d-9dbc-c21e087e66ed None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.285587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e84849-4d56-450f-b025-333c882f6b67 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.308761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.309651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.309927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.310194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.320177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.320759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:12.320863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.321103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.321392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.322176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.322479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.323843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:12.326173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.327231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:12.328074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 287/575] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.328771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.334924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 289/576] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.335753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.369542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.378668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.429010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933447-6e25-46b8-8c0a-a8f0cd975cdd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.437346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcffc6-9af9-4b7d-a2e9-a045b82fb2ae None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.463127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.463733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.464013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.464288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.465005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.465308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.466666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.469864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.470474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.470751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.471021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.471761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.472062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.473524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.478699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 288/577] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.479282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.485452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 290/578] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.486252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.577348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.580283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9233-7306-4fc5-952e-0d9988e4f650 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.586227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351845c-0739-4019-a335-fcfd8d17575f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.591118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.613960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.614561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.614835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.615111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.615847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.616148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.617627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.620125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.620766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.621073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.621347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.622051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.622354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.623697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.630002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 289/579] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.630779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.635077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 291/580] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.635867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.732744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370761-3ac3-49f5-8831-0105c4e5b89e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.733335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b509-768d-4f33-8e48-13273a8e1bdc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.765213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.765712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.765849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.766132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.766344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.766445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.766631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.766908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.774404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.775851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.777589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.779007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:12.787581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.788743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 292/581] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.790072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:12.793325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:12.794335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:12.795552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:12.795552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:12.795552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:12.795552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:12.795552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 290/582] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:12.796797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.809678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.821776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:12.891169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad556f52-d0fe-440c-bfc0-4651374788c6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.895146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960262-7713-4ad5-b95f-4c29f8733adb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:12.925925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.926584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.926865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.927139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.928150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:12.928789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:12.929066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:12.929343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:12.930118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=24f380d935524a2295fd9c01a8fec95b) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:12.930422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.931816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:12.937344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 291/583] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/role_assignments?user.id=24f380d935524a2295fd9c01a8fec95b&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:12.938105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:12.953174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=6268372f3956497aae05aec64d41e848, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:12.954673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.024706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 293/584] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:12 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:13.025764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.035800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.054289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bece4f-2f5e-4030-b707-bf8102fff93f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.098755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.099935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.100419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.101110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.114581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.116416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:13.134413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:13.135378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:13.136234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:13.136234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:13.136234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:13.136234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:13.143201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 292/585] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:13.145420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.146491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.189879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a02717-5c62-42d8-b8b4-8e1299fc1811 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.236605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.241816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.262428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 294/586] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:13.263230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.274979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8d5c4-a1b6-4b1e-bcf2-ef545ea7ae01 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.310875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.311639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.312017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.312404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.313230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:13.313634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.315093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.323271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.330615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 293/587] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:13.332089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.395199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad15ab6-298a-4743-b388-d267feea3acf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.422273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.436651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.437589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.437868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.439368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.439368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:13.439858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.441530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.456699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 295/588] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:13.459540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3081a25-1855-42d5-b71c-674a8ca149f7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.460443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.507752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.509224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.509672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.510086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.524061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.526021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:13.541712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:13.542807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:13.543755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:13.543755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:13.543755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:13.543755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:13.548999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 294/589] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:13.548999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.592482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b6c36-9879-4007-a1f7-b5056c6aa016 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.605771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.631666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.632435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.633481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.633481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.642923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.645278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:13.661653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:13.662718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:13.663604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:13.667755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 296/590] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:13.668697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.677085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250a38b-2aeb-43e6-b57d-3aa3c7acda35 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.717128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.725044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.725960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.726417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.726911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.737442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.739626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:13.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:13.755616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:13.756541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:13.757957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 295/591] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:13.758831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.791795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a077075-dd0a-4641-9311-9d8902978f8d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.837961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.838993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.839482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.840328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.841231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:13.841728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.843935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.859582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 297/592] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:13.860656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.871771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733779e-6f19-45d9-bf4e-9adc86f51b9a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.883621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:13.908128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:13.909211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:13.909326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:13.909835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:13.910919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:13.911332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:13.912923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:13.936751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 296/593] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:13.937719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:13.973845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7930-8bc0-4ce2-8c6d-e31e791e16f4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:13.975634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.011142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.012140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.012623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.013077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.014043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:14.014549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.016150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.034839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 298/594] 10.222.0.144 () {56 vars in 1144 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:14.035912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.044194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7f29-7354-47ae-9ae3-0a88034eeaad None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.082701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.083289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.083643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.083991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.091191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.092773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:14.104455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:14.105354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:14.106214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:14.110235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 297/595] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:14.111075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.140974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.152516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd02cfc-e9f3-4b3e-b6bd-dcd62da04aa6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.189537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.191081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.191569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.192065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.203057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.204842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:14.220691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:14.222172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:14.223045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:14.223045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:14.223045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:14.223045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:14.223258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af1415-6032-41a2-af3d-f8f7c9455961 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.225575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 299/596] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:14.226699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.244308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.270977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.271623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.271902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.272173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.285775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.288089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:14.312829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:14.313758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:14.314688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:14.314688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:14.314688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:14.314688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:14.314688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 298/597] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:14.315455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.347544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de6d2-52a3-420b-800c-0a6e187dad60 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.389889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.390670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.391031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.391385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.392215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:14.392613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.394157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.411904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 300/598] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:14.411904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.423420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.444939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c452d8-3b82-4ade-baef-cb3efabe1d26 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.490088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.491536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.492747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.493249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.493722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.494620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:14.495139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.497565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.513961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 299/599] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:14.519010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.530500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705077c0-bfcb-4be5-803b-0d8cf888995d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.599654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.601142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.601426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.601705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.602409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:14.602715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.604120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.616986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 301/600] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:14.617838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.644807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34b7c-fa54-4f8c-885d-cee085bca4f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.696246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.696931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.697213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.697493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/b6d652014d6446a0ae162120aac0ed94/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.710168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.714695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=b6d652014d6446a0ae162120aac0ed94, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.716318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.741522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738778-efee-42f1-a60e-b3c7aa8c9d9d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:14.773824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:14.784994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 300/601] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:14 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/b6d652014d6446a0ae162120aac0ed94/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:14.786398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:14.795010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:14.795977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:14.797662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:14.799571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:14.816156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:14.821022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:14.850076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:14.851207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:14.852406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:14.856282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 302/602] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:14.864751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.000556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d839-98f4-4117-be29-c0b722899216 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.003683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c1e28-f446-44e0-851a-ecfd596d2567 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.046596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.047227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.047510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.047783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.048705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:15.049008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.049441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.050481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.050600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8210e-0a79-4c3d-a963-9a6cc16fd318 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.050849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.051193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.058626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.060458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.065041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 303/603] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:15.067307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01170ef3-fd58-4cf0-a068-fa8573e85f49 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.067605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01170ef3-fd58-4cf0-a068-fa8573e85f49 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.067883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01170ef3-fd58-4cf0-a068-fa8573e85f49 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.069462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 304/604] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ee85adf9-161e-47d1-8e9f-412c06fca521 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:15.073119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:15.073993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:15.075574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:15.076454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01170ef3-fd58-4cf0-a068-fa8573e85f49 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.076598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 301/605] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:15.078961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.078961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.078961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.096942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:15.098229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:15.139802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:15.154491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:15.176476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 302/606] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:15.178015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.213956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01170ef3-fd58-4cf0-a068-fa8573e85f49 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.254062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.254638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.254920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.255200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/5361a7650f164ff8b11abfe6b3c4c382/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.270481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=5361a7650f164ff8b11abfe6b3c4c382, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.271982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891b16a-7c68-4947-b828-d46f3492c925 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.302001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46424801-b558-43cd-ae3a-130ebbfa1fff None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.338240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.338879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.339167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.339428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.340190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:15.340522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.341942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.354292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 303/607] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:15.355142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.367760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 305/608] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:15 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/5361a7650f164ff8b11abfe6b3c4c382/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:15.369822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483c1d7-d7bc-4dfd-b2bc-91ffac2c3315 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.370506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483c1d7-d7bc-4dfd-b2bc-91ffac2c3315 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.371195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483c1d7-d7bc-4dfd-b2bc-91ffac2c3315 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.373725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 306/609] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:15 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:15.374887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483c1d7-d7bc-4dfd-b2bc-91ffac2c3315 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.416230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:15.494694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb9835-4ef4-4ddd-993e-9c081faa7b92 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.514366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483c1d7-d7bc-4dfd-b2bc-91ffac2c3315 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.516357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:15.542013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.542733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.543017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.543289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.557712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-aac0a4a0-5689-45dc-8674-c74828161b08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:15.563835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:15.564808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:15.565647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:15.566256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.566256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.566256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.566460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 304/610] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:15.566538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/bd7e03b6ec884bf2bf5039aefba4e5e4/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.566615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.567107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.567350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.587313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:15.587512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=bd7e03b6ec884bf2bf5039aefba4e5e4, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.588561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:15.588940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.661962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 305/611] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:15.668407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.693554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 307/612] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:15 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/bd7e03b6ec884bf2bf5039aefba4e5e4/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:15.694745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.700621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:15.775776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0fcf3-b18d-48d2-996b-76570c5c3fb5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.843888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.844543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.844822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.845091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.845814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:15.846118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:15.848046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:15.858159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 306/613] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:15.869486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.890355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed29389-b8e6-4e1b-8336-efb14876b194 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:15.930322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:15.930988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:15.931270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:15.931539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:15.946741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:15.951702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:15.952629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:15.953432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 308/614] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:15.960865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:15.982657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.016380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.042795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984af6-bafa-4885-8872-3d287f04187a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.083471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977240a-6ba4-446a-8509-75bc4005a940 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.122591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.123242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.123525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.123798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.130381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.130995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.131274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.131545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.132265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:16.132600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.138815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.146731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.150333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.158629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 309/615] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:16.164721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:16.167349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:16.168216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:16.169109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:16.169109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:16.169109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:16.169109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:16.169109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 307/616] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:16.175062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.334776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.352031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55310-f90a-4219-9cfb-f836fe40ee02 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.399988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6387a-8999-4150-ab4a-0c9044de570c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.408686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.410111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.410391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.410662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.411364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:16.411667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.413237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.426310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 308/617] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:16.438988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.450191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.451125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.451651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.452150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.453476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.465244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.468956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:16.488153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:16.489270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:16.490181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:16.490181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:16.490181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:16.490181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:16.496500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 310/618] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:16.501150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.625163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d26b7-28b5-4df1-a3bf-b5b894a50176 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.664795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.665760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.666050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.666324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.674679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.676879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:16.696691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:16.697786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:16.698708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 309/619] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:16.704816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.716205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758fb59-fd6a-4a6f-8f4d-cb9ddacc8564 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.783407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.790828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:16.830494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bcb20-13ab-4da2-8f66-b9d23728356e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:16.832286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.833761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.834046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.834330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.835206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:16.835518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.845490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.874130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:16.874784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:16.875156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:16.875647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:16.876603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:16.876914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:16.879001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:16.881193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 311/620] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:16.900322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:16.906730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 310/621] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:16.913531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.026998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345129-1b07-4c68-a5d4-08631a97ec2e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.053683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.067893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.073446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.073446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.073446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.081059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.082802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:17.100861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:17.101807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:17.103122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:17.103122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:17.103122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:17.103122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:17.103122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 311/622] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:17.108065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.174208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ac45-ea81-4c04-b9e8-8d618e9e4efc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.223478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.224224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.224559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.224931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.232978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.235809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.242223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72402f05-745d-4e5f-a9c7-4ce597a4b2d3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8527a3ef-c587-4805-b576-170dc6e2bc40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:17.248692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:17.249649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:17.251170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 312/623] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:17.251773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166fbd-b7bd-4077-8a89-67692f95e20e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.251841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166fbd-b7bd-4077-8a89-67692f95e20e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.252119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57166fbd-b7bd-4077-8a89-67692f95e20e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.254234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 313/624] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:17.255699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57166fbd-b7bd-4077-8a89-67692f95e20e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.287890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.288809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.289095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.289363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.290232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:17.290532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.292123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d59eb7-dd38-4169-923d-9a636141ad6e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.307885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 312/625] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:17.309512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.309795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.310067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.322650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.333080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:17.337766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:17.366640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.384656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57166fbd-b7bd-4077-8a89-67692f95e20e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.413689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 313/626] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:17.420573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.426917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.427764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.428222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.428702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.429782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:17.430300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.432052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.473329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 314/627] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:17.478235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.561921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99528d7b-d719-462f-b52f-3ba41ecc3a6f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.599484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.600130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.600431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.600722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.604766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153269b-879c-4570-bf62-a2b609f4062f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.608146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.609616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:17.622176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:17.623087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:17.624028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:17.628119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 314/628] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:17.629084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.632767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.640742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.641382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.641711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.641921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.653313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:17.656606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:17.658301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:17.659956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:17.660812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 315/629] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:17.662355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.738895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3351c-2dec-4159-99d8-faeb0ead91db None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.772295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979009d2-b10a-4412-a12f-a22a7d8d592a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.778760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.779381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.779662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.779933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.815499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.816111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.816407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.816684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.817428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:17.817728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.819150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.831844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 316/630] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:17.832610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.858747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.860217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de139f05-7a73-41ae-9fd4-1c41bdc1abcb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.860688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.886339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:17.935968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f48d8-5920-43d1-ba4a-5dc0b4d6c036 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:17.967358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 315/631] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:17.968491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242039-f237-4e06-be4a-e3c3f8c1d893 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.968784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242039-f237-4e06-be4a-e3c3f8c1d893 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.969062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242039-f237-4e06-be4a-e3c3f8c1d893 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.970558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 316/632] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:17.971095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:17.971238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9242039-f237-4e06-be4a-e3c3f8c1d893 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:17.972281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:17.972794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:17.973461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:17.975591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:17.976135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:17.978480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:17.991445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 317/633] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:17.992537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.075958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9242039-f237-4e06-be4a-e3c3f8c1d893 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.086086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.099789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.104710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f26d-b775-42e2-8517-10c2058d9b2e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.110383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.111043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.111320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.114173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/24f380d935524a2295fd9c01a8fec95b/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.127529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=24f380d935524a2295fd9c01a8fec95b, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.129786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.140306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.148644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.148644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.148644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.152898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.154380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3bc3f1e5-d9b6-40e7-b9e0-39eaa6da8c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:18.168444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:18.169506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:18.170388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:18.170388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:18.170388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:18.170388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:18.170388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 318/634] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:18.171019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.171300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.171573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.189680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:18.191600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:18.201330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 317/635] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:17 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/24f380d935524a2295fd9c01a8fec95b/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:18.205631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.288505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 319/636] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:18.288505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.331411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc6b24-bdfb-4425-85f2-b76f7dff7498 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.367899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.371984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.372708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.372997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.373231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.384536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.385990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-85666862-a983-48de-b7f5-5c06d74b3303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:18.406016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:18.406959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:18.408612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:18.408612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:18.408612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:18.408612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:18.409469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609e1e8-22c7-4265-a1dc-72361dfa0d81 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.409557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 318/637] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:18.409557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64d56-6759-47b6-8e3e-6e15fc83ecda None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.409700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64d56-6759-47b6-8e3e-6e15fc83ecda None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.409888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b64d56-6759-47b6-8e3e-6e15fc83ecda None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.411291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 319/638] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:18.411829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b64d56-6759-47b6-8e3e-6e15fc83ecda None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.440154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.450184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.450806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.452258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.452258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.452258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:18.452545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.453837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.465986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 320/639] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:18.466695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.529142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b64d56-6759-47b6-8e3e-6e15fc83ecda None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.564968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.565552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.565827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.566090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.576860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b4d7c-a926-4fcf-b15c-d174c34164b1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.577273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ccab4f7c-ec3e-43ea-8490-879eec9926fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:18.583482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:18.584416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:18.585261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 320/640] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:18.586398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.586398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.586558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.604960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:18.606137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:18.608641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.617566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.618173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.623589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.623589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.623589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:18.623589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.623589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.638808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 321/641] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:18.639481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.677247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 321/642] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:18.678188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.705002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.766965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e281-7a3c-46b1-8bd0-a4686569fa17 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.787473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bfbfb-a4aa-41a9-821e-283bf7e727f5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.801619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.802271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.802536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.802812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.810098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.811578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.823463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:18.823722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:18.824786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:18.825652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:18.826075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.826075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.826075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.826075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:18.826075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.826962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.829528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 322/643] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:18.830217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.840902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 322/644] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:18.840902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:18.885383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.925317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:18.939526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811fad-afb3-47f9-a5d1-a8c468a2a00a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.947421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b468a00-2264-411f-b8b2-608646ee9cfc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:18.980664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.981659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.982136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.982580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.987888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:18.988548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:18.988833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:18.989108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:18.990971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:18.993048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:18.999464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:19.004823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:19.005734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:14:19.006811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:19.007852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:19.008998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:19.009810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:19.011349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 323/645] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:19.011503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 323/646] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:19.011652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.011804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.118012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a3111-dfc1-418f-9155-1c382ffecf1b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.124963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f335a-4ea6-47d2-8f87-f73204014ea1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.135050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.154214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.155042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.155427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.155797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.160071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.161065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.161664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.162360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.163375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:19.163500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.163842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.164909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.168481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.170075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.178060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 324/647] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:19.183523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:19.186347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:19.187336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:19.188379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:19.188379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:19.188379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:19.188379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:19.190578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 324/648] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:19.191460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.300258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba3c86-d78f-409c-babd-d3118a772579 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.300719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917a4a0-b21d-4096-b006-035a08e951d7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.336859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.337239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.337533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.337757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.337868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.338032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.338142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.338301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.338837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:19.339089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:19.339183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.339397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.340522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.340831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.348695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 325/649] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:19.349499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.363453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 325/650] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:19.364533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.396452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.411207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.462752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef03f-ac16-494b-b993-793f21517fba None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.475390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d60a-6d23-4b42-ac35-b1367b56b126 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.504061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.504712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.504990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.505375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.506171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:19.506478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.508022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.510656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.511336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.511689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.512031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.512745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:19.513050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.524609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.524858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 326/651] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:19.524858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.537466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 326/652] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:19.538355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.649597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.651423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2150e-e359-41a1-974f-bfd65fc21c78 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.670496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.670741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd246-40a7-421b-89e0-a8c79a7a6fd7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.692544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.693948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.694364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.694799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.707826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.709308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.714935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.715557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.715838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.716380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:19.726113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:19.727029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:19.727979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:19.727979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:19.727979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:19.727979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:19.727979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 327/653] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:19.728266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.805515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.807101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.847770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c91d-699b-4aa4-b272-4f501f5dd009 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:19.893871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:19.894813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:19.895270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:19.895712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:19.906927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:19.908602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:19.910386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:19.925497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 327/654] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:19.927244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:19.928543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:19.929422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:19.930328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:19.933945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 328/655] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:19.939455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:19.979187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.056695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaec51d-c2ad-4076-9ad9-b6a215e1e0ff None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.061705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af56e-7b3b-4c07-80bb-7979d0208de2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.092038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.092764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.093038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.093314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.104831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.106278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.107135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.108118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.108626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.109382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.110303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:20.110801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.113030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:20.121583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:20.122472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:20.123328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:20.123328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:20.123328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:20.123328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:20.127335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 329/656] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:20.128794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.130600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 328/657] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:20.131793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.188138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.232427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.244346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834637c6-0a18-4f35-9205-bd56761bf67a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.252317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335140-5c09-4f86-b316-772fe1262efd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.281401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.282011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.282286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.282555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.283334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:20.283545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.285458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.293545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.294537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.295019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.295428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.296374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:20.296836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.297172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 329/658] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:20.298008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.298905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affbb-e021-4548-bf1f-466c1f2b1290 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.324817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 330/659] 10.222.0.144 () {56 vars in 1142 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:20.326163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267f61-a658-449e-a948-c30642152ff2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.326460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267f61-a658-449e-a948-c30642152ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.326746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267f61-a658-449e-a948-c30642152ff2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.328016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 331/660] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:20.328844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc267f61-a658-449e-a948-c30642152ff2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.412485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa80c-34ce-48fe-b301-3f512b0cab19 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.420691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.440719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc267f61-a658-449e-a948-c30642152ff2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.454351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.454986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.455265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.455565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.463354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.464807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:20.477144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:20.478350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:14:20.479477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:20.482914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 330/661] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:20.483710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.484294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.485018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.485301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.485586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.494556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.496075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.498052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e676037f-4f8c-4b37-b34a-c64cf49a98f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:20.509442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:20.510311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 332/662] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.515988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.516619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.531854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:20.533120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:20.601938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90adf-a185-4b44-9994-77074912ad02 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.604604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 333/663] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:20.605434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.655918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.656955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.657352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.657525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.693537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.726751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.727549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbc707-f3aa-4a52-b356-364bada21c94 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.728206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.756617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.766187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.766848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.771764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.771764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.771764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:20.771764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.771764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.787990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 334/664] 10.222.0.144 () {56 vars in 1144 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:20.788703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.852027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 331/665] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:20.852027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.897453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301faee-5be2-472e-b6f5-3d82e61c3764 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.916723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:20.931869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.932596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.932957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.933298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:20.934370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:20.934740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:20.936207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:20.953125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 335/666] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:20.953125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:20.958025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f149d9-4eaf-4756-a6f8-b21fc00d27c8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:20.995152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:20.995776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:20.996054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:20.996330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.003375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.007159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:21.011971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:21.012928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:21.013786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 332/667] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:21.014480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.061428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc023e-5667-4747-b325-4a205bd94cb1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.095192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.095797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.096653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.096965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.119039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd4593-16f1-4114-a1e7-4744628497b9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.148105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.160054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.160792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.161107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.161385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.168354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.170075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.172252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.173886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:21.192696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:21.193811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:21.194909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:21.194909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:21.194909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.194909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:21.196840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 333/668] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:21.197763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.223319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.282997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 336/669] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:21.282997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.303176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2ea1b-f1cb-4f53-b1c7-5543f8d25277 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.339015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.339623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.339901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.340170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.340983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=b6d652014d6446a0ae162120aac0ed94) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:21.341481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.342692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.356428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 334/670] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/role_assignments?user.id=b6d652014d6446a0ae162120aac0ed94&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:21.356428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.406799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c65cb-a4e9-41cf-8395-abc3a6526658 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.407445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.457536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.458174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.458459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.458735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.459465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:21.459774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.461757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.476661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf3933-12ac-4928-b748-8f138ce9f7df None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.478967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 337/671] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:21.480730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.499210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.523147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.524152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.524651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.525114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.525996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:21.526509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.528085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.542290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 335/672] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:21.543150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.623461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e764f-7f3c-4bfe-b569-e3334dfc9946 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.666859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f67d5d-316d-4dce-b676-dc670733e2d0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.668913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.680256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.681020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.681376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.681723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.696867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.698868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.704243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.705119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.705605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.706052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.716700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:21.717495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:21.718319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:21.719181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:21.719181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:21.719181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.719181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:21.719181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 338/673] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:21.720711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.725098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:21.740636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:21.741666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:21.742738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:21.742738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:21.742738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:21.742738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:21.744578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 336/674] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:21.745426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.792736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:21.838617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6404e-0ed4-40da-a096-2352e2ff7c32 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.847671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192415c-3511-4579-861b-9e6fc32d39a2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:21.875023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.875773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.876050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.876466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.881193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:21.881808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.882087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.882359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.883045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:21.883349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.883534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:21.884738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a900aa-7c59-4a2a-a513-f2ab8ca7f0b4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.884997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:21.903865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 337/675] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:21.904051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:21.904954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:21.905873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:21.906288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837053c8-4475-4688-bfef-042ddbaaf21c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:21.906288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837053c8-4475-4688-bfef-042ddbaaf21c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:21.906288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837053c8-4475-4688-bfef-042ddbaaf21c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:21.906471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 339/676] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:21.906471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.906589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 338/677] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:21.907561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837053c8-4475-4688-bfef-042ddbaaf21c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:21.925011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.020978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8fdc-ae66-4ec9-a9ea-7d1137b5c32d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.030720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837053c8-4475-4688-bfef-042ddbaaf21c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.044198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.059600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.060216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.060531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.060805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.061586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.061798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.063165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.067610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.068268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.068600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.068875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.069628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.069934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.071348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706570fd-4f70-461a-987d-f3bc5f35654b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.076327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 340/678] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.076986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.088319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 339/679] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.094110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.094400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.094700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.114212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:22.115411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:22.177610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae101-e353-475d-9f19-2e64caaf28c6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.192847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 340/680] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:22.192847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.195579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.219030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.219811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.220092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.220471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.228543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.230017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:22.241448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:22.242490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:22.243359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 341/681] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:22.243834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.289337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.319590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b66c69-61a5-431b-90de-4a3fdb2a789b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.354245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb40e95-46bd-4200-b6a4-b887658b988c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.354612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.355249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.355565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.355803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.362822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.364270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:22.376739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:22.377905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:22.378809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 341/682] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:22.379399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.394202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.394811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.395098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.395371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.396165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=5361a7650f164ff8b11abfe6b3c4c382) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.396512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.397951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.404023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 342/683] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/role_assignments?user.id=5361a7650f164ff8b11abfe6b3c4c382&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.404892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.477795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.526236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69b880-1059-4ff2-8ebc-dabbd60941b6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.528812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.543872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717bc6-3018-470a-839b-54f70bc72f9b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.565568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.566099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.566381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.566653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.567493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.567713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.569248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.584425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 343/684] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.586080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.596572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.597225 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.597503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.597775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.610606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:22.617095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:22.618052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:22.618859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 342/685] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:22.619570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.721600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf77b7-6a26-4497-aa86-679f5129c09b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.761079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3578-d189-444c-a0bd-9c8fe7852273 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:22.762018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.762806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.763198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.763644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.764652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.765066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.766611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.790932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.791259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 344/686] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.793116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.807545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:22.807545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:22.807545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:22.807545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:22.807933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=bd7e03b6ec884bf2bf5039aefba4e5e4) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:22.807933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:22.807933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:22.816468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 343/687] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/role_assignments?user.id=bd7e03b6ec884bf2bf5039aefba4e5e4&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:22.817548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:22.831629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:22.976424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425e296-a280-4aa0-81b9-0bc38f9343da None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.022775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5ca94-86ee-4191-9569-adfbf8fe53de None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.028197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.028855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.029136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.029409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.030135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:23.030445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.031825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.047236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 345/688] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:23.052832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.096804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.096804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.097073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.097323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.165706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.173526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.185008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b36c10-fa09-42df-8ec6-1d8087eb5bdc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.189888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.191367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.224932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.225956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.226436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.226906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.241560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.243459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:23.259637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:23.260586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:23.261465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:23.261465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:23.261465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:23.261465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:23.263637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 346/689] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:23.275095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.351872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 344/690] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:22 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:23.361446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.415154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47c01-e537-4ec8-9e00-84d7cd005967 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.454901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.456085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.456368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.456669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.458554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:23.458864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.460227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.467317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.483216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 347/691] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:23.489292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.525435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.538229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76834044-f237-4d58-8e16-95359c3d97d6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.576054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.576742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.577130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.577417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.588694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.590274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.613543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117402-15aa-42eb-bed7-8e15ad968dde None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.653051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.653719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.653991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.654270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.663835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.665404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f4d58db0-8708-4f6f-8fef-ada249a3e7e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:23.678966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:23.680331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:23.681554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 348/692] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:23.682157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49a32-03de-44ce-b4b8-e9b06cb0bef3 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.682658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49a32-03de-44ce-b4b8-e9b06cb0bef3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.682812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49a32-03de-44ce-b4b8-e9b06cb0bef3 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.684558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 349/693] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:23.690463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.690784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.691004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.711726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:23.713015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:23.738352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:23.741935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 345/694] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:23.744052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.777768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.800544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 350/695] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:23.801452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.849311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7614c4a-cdd3-4cd4-a09f-3d3497396223 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.886290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.887282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.887740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.888185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.889173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:23.889658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.891407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.901211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:23.911193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 346/696] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:23.912277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:23.927783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567501-4b51-4a78-a1c1-3de53efdf476 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:23.965366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:23.966250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:23.966675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:23.967094 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:23.968023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:23.968497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:23.970054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:23.988025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 351/697] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:23.989151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.017019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ac64-3be5-471f-96c5-b7d8519125dd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.039697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.055508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.060937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.060937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.060937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.070350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.072051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:24.087710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:24.088772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:24.089637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:24.089637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:24.089637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:24.089637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:24.091698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 347/698] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:24.092756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.101856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef15c-b853-4930-a597-76b1346851c6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.137714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.138256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.138545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.138827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.144592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.194450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdfb7d-e7bf-4622-b207-92e97a492927 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.211977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.213570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.231651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.232762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.233245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.233715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.241333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.243111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:24.255809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:24.256713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:24.257596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:24.260094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 348/699] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:24.261177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.265929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.324561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 352/700] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:24.325516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.371602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a85c-c54c-4fe3-bb43-352c5eb56015 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.383812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.408328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.409426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.409964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.410438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.411401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:24.411920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.413638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.428119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 349/701] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:24.429268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.440244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3b959-1985-449e-9392-ac01aa17683e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.482185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.482810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.483091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.483364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.484118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:24.484449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.485870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.510378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 353/702] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:24.511550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.521230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.541512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5824a-b4cc-4708-8ef3-a707d6764f18 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.577200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.578192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.578696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.579167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.589770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.591476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:24.606451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:24.607347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:24.608232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:24.608232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:24.608232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:24.608232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:24.610389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 350/703] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:24.612361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.617133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e7c-4ccb-40a4-844a-32750a1943b9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.640251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.653985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.654605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.654888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.655164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.655852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.657721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.716126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe66d2-b6b1-472d-ade0-f58e9be5af23 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:24.733776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 354/704] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:14:24 2022] POST /identity/v3/users => generated 255 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:24.734774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:24.772149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:24.773165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.773655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.774145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.775268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=24f380d935524a2295fd9c01a8fec95b) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:24.776606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:24.778639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d86b5a-1a46-4146-b07a-9d426148210f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:24.796664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 351/705] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/role_assignments?user.id=24f380d935524a2295fd9c01a8fec95b&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:24.796664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fba12-e188-4c93-9a96-58cf65f3c371 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.796664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fba12-e188-4c93-9a96-58cf65f3c371 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.796664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fba12-e188-4c93-9a96-58cf65f3c371 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.807083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 352/706] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:24 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:24.810769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:24.811104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:24.811370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:24.849909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:24.851156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:24.859789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:24.980265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 353/707] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:25.008716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.036925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.055524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32771-10ec-4f3d-ad28-e11205d6ae5f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.100582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.101714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.102191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.102660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.103576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:25.104074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.105683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.122222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 355/708] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:25.130494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.160554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6695d1-ce9f-4abc-98fb-b3f26aa75ac9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.212745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.213965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.214347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.214871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.215677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.217837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:25.233208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:25.234202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:25.235177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 354/709] 10.222.0.144 () {56 vars in 1134 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:25.240785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.264795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c722e-7c58-4118-8735-4bec9e1be079 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.292725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.319637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.320400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.320705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.320975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.334093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.336129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-46c02488-7073-4f4a-b607-3ee389c2131e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:25.348736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:25.349705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:25.350663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 356/710] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:25.351626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724da8-fc74-4d2d-990c-9d7d6786d1ba None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.351626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724da8-fc74-4d2d-990c-9d7d6786d1ba None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.351900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724da8-fc74-4d2d-990c-9d7d6786d1ba None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.357266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 357/711] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:25.358120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724da8-fc74-4d2d-990c-9d7d6786d1ba None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.364258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.385881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e59474-9172-4528-9284-1b03dcb61b9d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.424107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479536e8-c291-44ab-9226-154001954e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.424861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479536e8-c291-44ab-9226-154001954e39 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.425241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479536e8-c291-44ab-9226-154001954e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.425665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479536e8-c291-44ab-9226-154001954e39 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.426409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479536e8-c291-44ab-9226-154001954e39 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:25.426839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479536e8-c291-44ab-9226-154001954e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.428614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479536e8-c291-44ab-9226-154001954e39 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.435956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 355/712] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:25.438423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.438423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.438423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.460782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:25.462339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:25.487336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724da8-fc74-4d2d-990c-9d7d6786d1ba None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.528454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.529270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.529549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.529822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.530571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:25.530934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.532552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.540872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 356/713] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:25.542069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.551144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 358/714] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:25.553855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.611605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.642632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.676483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b7aec-028f-4e51-a03a-3e2301620be6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.682016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b71260-a26f-4644-a995-5600b463bfa4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.718544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.719157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.719889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.719889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.720706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:25.721609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.722970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.736415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 359/715] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:25.736415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:25.737235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.737235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.737235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.737235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.737590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:25.737590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.740002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.757493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 357/716] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:25.913201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de47ea-69fe-4ba0-8fe9-6c1bd4a0dbc8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:25.922581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:25.953881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:25.954621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:25.955068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:25.955962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] PATH_INFO: `/v3/services/network` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:25.956646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:25.958017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:25.970464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:25.972066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:25.972923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 360/717] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:25.980102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:26.095669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae507b-0cdf-4c2b-8536-66f3b7f466e5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:26.131074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:26.131702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:26.131964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:26.132233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:26.133038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] RBAC: Adding query filter params (name=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:26.133342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:26.134701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a83fa-f48f-4cdb-a75f-8e673241fd5e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:26.145633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 358/718] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:25 2022] GET /identity/v3/services?name=network => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:26.147407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:26.271448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80385d7-6637-4250-9a53-781a028c2b92 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:26.319120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:26.319844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:26.320121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:26.320415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:26.321112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] RBAC: Adding query filter params (type=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:26.321415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:26.322905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01854f3e-2527-4cdc-b987-6c4669431dcc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:26.333265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 361/719] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:26 2022] GET /identity/v3/services?type=network => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:27.257425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf864-7334-4ed6-a9f6-980b153759b1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.258441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf864-7334-4ed6-a9f6-980b153759b1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.258945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaf864-7334-4ed6-a9f6-980b153759b1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.260720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 359/720] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:27.266177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.266992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.267473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.274913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503499ea-c5b4-450d-ba5f-726b8efc2b1a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.275205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503499ea-c5b4-450d-ba5f-726b8efc2b1a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.275480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503499ea-c5b4-450d-ba5f-726b8efc2b1a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.277060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 360/721] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:27.282020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.282307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.282587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.296407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:27.296920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:27.298131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:27.316180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:27.317399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:27.360644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:27.426518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 362/722] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:14:27.434809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 361/723] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:14:27.452564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:27.485523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:27.625518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae89be-ba94-4190-b656-6b91d67113c3 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:27.654488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5499dde-47fb-43ef-907b-91ff76a64bba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:27.689313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:27.690839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.691294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.691739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.708138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:27.710681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:27.717275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:27.718178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:27.718982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:27.724196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:27.725059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.725481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.725891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.729346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 363/724] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:27.733571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:27.750384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:27.755683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:27.756854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:27.757939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:27.760168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 362/725] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:27.765781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:27.792151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:27.903790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66876-08de-47a8-9622-485f01447090 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:27.949518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:27.950146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:27.950430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:27.950705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:27.951428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:27.951734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:27.953162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:27.975824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 364/726] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:27.977938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0ecd-46f9-484a-9359-f3ef030ae53f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:27.980825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.053858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.054743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.055222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.055640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.056745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:28.057194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.058724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.069221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 363/727] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:28.077336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.116735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.129319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae95fc-cf4e-4623-80dd-51897762eeae None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.166663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.167824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.168594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.169041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.170590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.190582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.192642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c61ed07f-a482-4295-a091-20070f334416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:28.213354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:28.214484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:28.215557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:28.215557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:28.215557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:28.215557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:28.218107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 365/728] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:28.218989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09565e8c-2c23-426c-b19e-e7e595fd10ba None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.219542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea937-1045-4e5c-bf08-e06da0cc2f4c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.219829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea937-1045-4e5c-bf08-e06da0cc2f4c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.220108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea937-1045-4e5c-bf08-e06da0cc2f4c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.224466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 366/729] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:28.225455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaea937-1045-4e5c-bf08-e06da0cc2f4c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.260559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75485a49-b885-4909-9482-307935c16d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.261872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75485a49-b885-4909-9482-307935c16d35 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.262378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75485a49-b885-4909-9482-307935c16d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.262890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75485a49-b885-4909-9482-307935c16d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.274832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75485a49-b885-4909-9482-307935c16d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.276718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75485a49-b885-4909-9482-307935c16d35 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-75485a49-b885-4909-9482-307935c16d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:28.292374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:28.293291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:28.294139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:28.294139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:28.294139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:28.294139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:28.296296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 364/730] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:28.297753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.298235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.298722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.318646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:28.320187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:28.354775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaea937-1045-4e5c-bf08-e06da0cc2f4c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.398363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 365/731] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:28.399557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.409580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.410216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.410500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.410777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.412526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:28.412845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.414233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.424149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.430048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 367/732] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:28.431975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.456497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.544915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efbdb2-20ac-4663-8c39-66ed9c2159dd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.566297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc93cc9-c025-4599-b1f3-ccfcf6958f6f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.582072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.582487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.582766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.583035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.583722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:28.584020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.585412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.596892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 366/733] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:28.597675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.607561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.608539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.609055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.609556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.620949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-04ff5f96-0f93-4917-a3f4-9d7a6511152a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:28.626600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:28.628042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:28.628876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:28.631053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 368/734] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:28.632401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a512-0386-470d-a02d-915ecc49e4c1 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.632833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a512-0386-470d-a02d-915ecc49e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.633253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a512-0386-470d-a02d-915ecc49e4c1 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.634638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 369/735] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:28.635484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a512-0386-470d-a02d-915ecc49e4c1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.695628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.701689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae4d20-128f-4ad4-b4b9-b3f4df7e92b7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.703715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.740529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.741176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.741458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.741732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.746427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a512-0386-470d-a02d-915ecc49e4c1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.750245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.751688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:28.763348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:28.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 367/736] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:28.765292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.788251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.788946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.789229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.789504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.797337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.798767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b4b5e49a-639a-4e2f-8e74-d125e6b4ab01 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:28.811371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:28.812309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:28.813292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:28.817272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 370/737] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:28.818381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.818670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.818949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.836876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:28.838062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:28.874083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c61ca-c53b-4215-b3c1-041edb8d9a17 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:28.905450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 371/738] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:28.906285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.917284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:28.917967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:28.918171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:28.918442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:28.919154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:28.919497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:28.920923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:28.930412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 368/739] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:28.931232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:28.950493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:28.962302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.011028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe62e74-e244-46f1-be50-7df7c668fc92 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.028947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341da5a-495b-4ab6-a4af-ac5e1fbdfca4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.044009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.044651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.044938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.045209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.046006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:29.046311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.047732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f02af7-ecbc-4f9b-b5d0-22f49fd950f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.063834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.064476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.064756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.065027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.065807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:29.066110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.067780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 372/740] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b83e4-e7dd-45d7-8da0-02d8d337bfd6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b83e4-e7dd-45d7-8da0-02d8d337bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b83e4-e7dd-45d7-8da0-02d8d337bfd6 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 373/741] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:29 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:29.077367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b83e4-e7dd-45d7-8da0-02d8d337bfd6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.090809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 369/742] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:29.091623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.169626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.179390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b83e4-e7dd-45d7-8da0-02d8d337bfd6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.187042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.192849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dcc8b-b82e-41bf-81be-ba29e83aca73 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.213725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.214272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.214552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.214825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.215512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.216944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd128-c308-47b2-8972-6da576d185f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.226863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.227495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.227802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.228051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.238645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.240131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:29.254425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:29.255323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:29.256182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:29.256182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:29.256182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:29.256182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:29.256182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 370/743] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:29.256500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.309194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 374/744] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:29 2022] POST /identity/v3/services => generated 252 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:29.310418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.310702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.310976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.329676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:29.330901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:29.376133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe96650-3648-45ad-b9fb-040ea206af62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.411075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.432416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 375/745] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:29 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:14:29.432416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.435948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.436619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.436902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.437177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.524198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.525703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.539226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.548070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6405d-5768-46a3-93d9-27cc9dbac010 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.585888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.586543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.586827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.587091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.636123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 371/746] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/auth/tokens => generated 904 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:29.636993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.665057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.666619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.666754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.749497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0bd-2529-423c-9bd2-39f57194b295 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.768420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.782327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 376/747] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/auth/tokens => generated 904 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:14:29.784128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.785268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.785889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.786169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.786443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.787133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:29.787438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.788870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.801669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 372/748] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:29.802484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:29.890742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:29.894668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42cb40-3d01-4b79-a873-8965e4bff70c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.909234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8dcbc-1799-459c-bd09-abcd02535716 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:29.935684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.936645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.937191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.937477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.938227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.941087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.946468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:29.947434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:29.947823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:29.948309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:29.963964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:29.966274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:29.986172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:29.987164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:29.988185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:29.988185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:29.988185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:29.988185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:29.988185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 373/749] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:29.989306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.003615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 377/750] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:29 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:30.005255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.087487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.126636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377cc0d-a0d6-4731-b10f-78b16c8ed67b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.157743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8f369-238b-4a78-abd9-006ae3d6da91 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.165490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.166488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.166947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.167399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.170714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.179486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.181298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:30.197471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:30.198473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 374/751] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:30.203975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.204303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.204303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.204303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.204303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.214547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.216004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:30.230101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:30.231024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:30.231925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 378/752] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:30.233132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.343447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10caeb-5f78-4b50-9332-efa2e926a9d9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.344083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e371bf-e991-47ac-bcf9-a0115546ce68 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.354486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.381534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.382696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.383479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.384043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.385042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:30.385548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.386410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.387027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.387312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.387586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.388278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:30.388588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.388716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.389952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.410256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 379/753] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:30.410256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.410503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 375/754] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:30.413545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.469568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.524971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be24f-9e27-4f53-a56b-f1bec495b7c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.555734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777a66d-352b-43a7-9d28-f8008b091716 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.571148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.572228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.572752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.573238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.580627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.582329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.593576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.594126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.594404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.594676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.595467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:30.595770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:30.596511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:30.597336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:30.598357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:30.598775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.600839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 376/755] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:30.601917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.621361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 380/756] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:30.622275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.651805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.707628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095e85b-02b3-4884-8d05-960a862a0440 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.717158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.731866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603a62-e674-42ee-9914-d1b2857813e6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.744066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.745054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.745544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.746051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.753373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.755060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:30.767436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:30.768321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:30.769220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:30.769653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.769653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.769653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.769653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.769914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:30.770500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.771954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.777158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 377/757] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:30.778224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.785972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 381/758] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:30.786902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.881442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:30.886932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e51cb-f416-4571-baff-320e43ac504d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.897532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236bc8c-05e9-4b55-982f-0fc06d2b85a5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:30.923305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.924321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.924852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.925330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.933344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:30.934098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:30.934383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:30.934655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:30.935400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:30.935704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.937247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.938671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:30.940882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:30.955603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 382/759] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:30.955603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:30.961381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:30.962288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:30.963181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:30.963181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:30.963181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:30.963181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:30.966505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 378/760] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:30.967921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:30.990617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:31.065771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bdab6-d60b-4913-9eaf-4c2c0d93cec0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.073469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef16-53aa-4b76-80ff-2dccf299f064 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.105631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.106241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.106519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.106788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.107571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=c0f8e4e0592447ef8cc91842386e25bd) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:31.107892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.109309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.111958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.112842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.113327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.113799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.114794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=24f380d935524a2295fd9c01a8fec95b) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:31.115306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.115899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 383/761] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/role_assignments?user.id=c0f8e4e0592447ef8cc91842386e25bd&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:31.116782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.117504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.128529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 379/762] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:30 2022] GET /identity/v3/role_assignments?user.id=24f380d935524a2295fd9c01a8fec95b&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:31.131755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:31.237532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eacb46-dfeb-4442-9636-7b8b17074b55 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.275931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.276730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.277007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.277323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.278034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:31.278337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.279706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d4d8-1107-40b8-a80a-32258519de3f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.297627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 384/763] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:31.301558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.421001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:31.437680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f94f7-fbcf-4776-a1da-f037cd8a86b9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.465798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac102f-2c4e-48df-ac98-10347da0685d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.466104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac102f-2c4e-48df-ac98-10347da0685d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.466349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac102f-2c4e-48df-ac98-10347da0685d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.467821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 385/764] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:31.474294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.474727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.475146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.485632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.486294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.486570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.486838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.498094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.500676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.512975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:31.514465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:31.515195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:31.516029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:31.518194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:31.528511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 380/765] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:31.528511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.652789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:31.657911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 386/766] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:31 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:31.682853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.725191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2c6a-c62e-4cde-b53b-8b37f5d4fe90 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.762091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.762716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.763003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.763294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.764056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:31.764369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.769431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.803786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 381/767] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:31.813368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.823474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:31.843783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c08d-e5a9-4953-a4c4-abb0698d37a7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:31.897143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:31.898430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:31.899039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:31.899512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:31.900340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:31.902062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:31.924770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:31.925729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:31.926525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:31.929969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 387/768] 10.222.0.144 () {56 vars in 1134 bytes} [Tue Jul 26 16:14:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:31.934389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:31.976705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d378-2104-4fb7-a0ff-9f4fb5de4183 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.015502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.016175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.016487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.016768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.020858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=6268372f3956497aae05aec64d41e848, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.022302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.054398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:32.099278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c727aa8-f8d0-4970-8890-fed0ed15ee32 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.120414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 382/769] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:31 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:32.135922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.136978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.137458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.137992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.138914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:32.139428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.141050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.150716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 388/770] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:31 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:32.157012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:32.283549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:32.325437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2ffc5-90a4-4eb5-830b-55b36d4c8390 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.364774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.365398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.365683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.365961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.366645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:32.366950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.368317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dc025-6c67-4480-92e5-266653f81aea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.373697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 383/771] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:32 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:32.379798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:32.453964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9965a2-c5ed-4b01-b5fa-ff8e09805a94 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.454503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9965a2-c5ed-4b01-b5fa-ff8e09805a94 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.454503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9965a2-c5ed-4b01-b5fa-ff8e09805a94 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.456470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 384/772] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:32.470935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.471219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.471489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.506936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:32.508151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:32.513948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd31-ae1a-4ca1-a914-7a420aac85a8 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.562563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:32.576771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.577394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.577674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.580562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.580562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] RBAC: Adding query filter params (interface=public, service_id=1b62332df9ad42daa7ea141338bc664c, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:32.580562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.581019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.601266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 389/773] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:32 2022] GET /identity/v3/endpoints?service_id=1b62332df9ad42daa7ea141338bc664c&interface=public®ion_id=RegionOne => generated 195 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:32.609467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:32.654965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 385/774] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:32 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:32.685481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:32.747026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:32.758107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e1ec-9ba7-408a-8476-09dd6d2e4a20 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.790372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8247-9a7c-439d-91c0-e8ad603b101a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:32.830260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.830871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.831159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.831429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] PATH_INFO: `/v3/services/network` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.832057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.832291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:32.832933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:32.833212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:32.833459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.833559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:32.834123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:32.835577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:32.842757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:32.843702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:32.844535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 386/775] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:32 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:32.845295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 390/776] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:32 2022] GET /identity/v3/services => generated 776 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:32.850177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:33.001861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e342d5-dedb-4539-9bda-6d10d8c9052c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:33.036937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:33.052517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:33.054697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.054697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.054697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.054697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] RBAC: Adding query filter params (name=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:33.055166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:33.056580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:33.062034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 387/777] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:32 2022] GET /identity/v3/services?name=network => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:33.069289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:33.136638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:33.196647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9239-29a2-4aac-b171-d2053eb4ac5d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:33.240130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:33.240772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.241050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.241321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.242006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] RBAC: Adding query filter params (type=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:33.242307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:33.243670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:33.249266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 391/778] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:33 2022] GET /identity/v3/services?type=network => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:33.254899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:33.398751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a1c3-cb79-4762-8195-a5e083c76a98 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:33.445380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:33.445994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.446271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.446540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.447280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] RBAC: Adding query filter params (interface=public, service_id=a0e43d7720f548988a64b31c0be27633, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:33.447585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:33.448965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9dd34-1c7d-4e44-8725-7fdc7823a416 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:33.457386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 388/779] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:33 2022] GET /identity/v3/endpoints?service_id=a0e43d7720f548988a64b31c0be27633&interface=public®ion_id=RegionOne => generated 195 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:33.463145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:33.619106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416d43-7530-45b2-a439-3ea8c37d0176 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:33.648147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:33.675341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:33.675341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.675341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.675341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.675732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:33.675732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbdc5-57ee-4b46-90b7-768a487facfb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:33.680143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 392/780] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:33 2022] GET /identity/v3/services => generated 776 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:33.699008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300c28-af85-4cc9-bb2c-7cb994d2460e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.699327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300c28-af85-4cc9-bb2c-7cb994d2460e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.699581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300c28-af85-4cc9-bb2c-7cb994d2460e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.701749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 389/781] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:33.712066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.712409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.712668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.722232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3aba2-177f-4cf8-a966-a00b183231eb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.722845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3aba2-177f-4cf8-a966-a00b183231eb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.723286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3aba2-177f-4cf8-a966-a00b183231eb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.725082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 390/782] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:33.730649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:33.731199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:33.731704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:33.746109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:33.747663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:33.751461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:33.753485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:33.854362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:33.873053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 393/783] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:33 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:33.898048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 391/784] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:33 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:33.910890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:33.922630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.041413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0561b2-a421-4a41-8cee-bd88fb47c270 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.046632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4facd-e8a3-467b-ac59-cd8c5bd95b5a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.057907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.088868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.089868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.090483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.090993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.099231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.100165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.100922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.101431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.113002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:34.115374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:34.118444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:34.119350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:34.120180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:34.122925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 392/785] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:34.128342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:34.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:34.130124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 394/786] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:34.132261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.135630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.202304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.238181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754643e9-6ef1-4ea4-9837-dd7dd08b39e9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.277673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.278707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.279197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.279689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.280826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:34.281358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.282985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.291170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec49d-40cd-48e1-8e83-e11c3ce8cd0d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.297210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 393/787] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:34.298371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.345257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.345777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.346060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.346350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.347123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:34.347366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.348769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.374665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 395/788] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:34.385394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.406402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfa8d-067e-4360-bd86-116876c8939d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.424771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.445691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.446711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.459579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.459579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.460346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.460984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.462650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:34.484055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:34.485214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:34.486704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:34.486704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:34.486704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:34.486704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:34.489268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 394/789] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:34.496284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.597048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27118e-496d-44cd-bf17-9e595109ea4a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.659598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.659598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.659872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.665088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.671298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.672778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.680352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd743fe0-d2dc-4282-8204-67abc502b97b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:34.688607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:34.689696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:34.690549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:34.690549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:34.690549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:34.690549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:34.690549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 396/790] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:34.698396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.726215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.726859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.727190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.727475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.728271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:34.728611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.730082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830df0c-2577-49a0-9e45-de2b0a9bdbf3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.746902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 395/791] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:34.751729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370ee4-7325-4d24-88af-f6824fd2b774 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.752012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370ee4-7325-4d24-88af-f6824fd2b774 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.752282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370ee4-7325-4d24-88af-f6824fd2b774 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.753872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 396/792] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:34 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:34.756024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90370ee4-7325-4d24-88af-f6824fd2b774 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.773871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.807768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:34.821526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7e146-c81a-4a40-87dc-35967814edab None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.862340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.863321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.863826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.864301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.865375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:34.865888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.867580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee55e7e-3b40-47cf-ba3f-2ba793141e81 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.879353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90370ee4-7325-4d24-88af-f6824fd2b774 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:34.882225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 397/793] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:34.884011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.884565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.885060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.904261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:34.905769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:34.924867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:34.925504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:34.925881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:34.926380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:34.936658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:34.938851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:34.953239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:34.954178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:34.955684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 397/794] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:34.956254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:34.990032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 398/795] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:34 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:34.991493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.062690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.086693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.101061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4ecb-6d4d-4af3-b156-fab80379311d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.111371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ad30c-27b8-4a74-b38a-0685f9f82dd6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.140508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.141497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.141953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.142891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.150334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.151059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.151415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.151764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.152723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:35.152902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.153213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.154756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.154897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:35.170286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:35.171288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:35.172198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:35.179456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 398/797] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:35.180594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.181302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 399/797] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:35.182229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.289195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e62416-448e-40c4-a533-f6ead922f888 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.292427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eba59b-eb8d-4002-b215-9eab979fe067 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.325731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.326830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.327398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.327953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.330445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.330918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.332093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.332590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.332767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.335551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.342375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:35.343983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.345497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:35.348147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:35.352408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 399/798] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:35.353251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:35.361085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:35.361950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:35.363166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:35.363166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:35.363166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.363166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:35.363166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 400/799] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:35.363434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.466755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5564b-e81a-4259-9777-109d4134799f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.487574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f81b-01c7-41d6-90dc-a89c35983002 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.511805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.512619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.512905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.513179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.513984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:35.514302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.515723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.526295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.526912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.527197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.527473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.528190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:35.528528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.529884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361e5cf-d054-429e-865f-6506da4de38e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.538266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 400/800] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:35.538266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 401/801] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f9331-6459-4d82-b2b4-6ba00cb76acf None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f9331-6459-4d82-b2b4-6ba00cb76acf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f9331-6459-4d82-b2b4-6ba00cb76acf None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 402/802] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:35 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:35.546800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f9331-6459-4d82-b2b4-6ba00cb76acf None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.596463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.603183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.643805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeaac0-5775-4389-846b-5d87bf69dc2c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.653145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f9331-6459-4d82-b2b4-6ba00cb76acf None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.679411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.680031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.680317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.680622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.681319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:35.681633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.682998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.691821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.692934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.693385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.693660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.695410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 401/803] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:35.700752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.704684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.706139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d27b8992-f4df-44f2-99f4-5cf70b6e81d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:35.721156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:35.722042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 403/804] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.727947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.744552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:35.745850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:35.809246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b920756-10d7-420a-b09f-a2d69b06b130 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:35.829367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 404/805] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:35 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:35.829367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.829621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.857037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:35.857718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:35.858238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:35.858590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:35.871762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:35.873360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:35.880093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:35.890072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:35.890987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:35.891852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:35.891852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:35.891852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:35.891852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:35.895797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 402/806] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:35.896765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:35.967133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215b935-324b-4dd0-8929-8b3407d3219e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.002435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.003293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.003749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.004024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.012156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.013217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684325d1-a716-4700-bbe6-32a86a8c4e62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.013627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:36.025549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:36.026454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:36.027351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 405/807] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:36.027853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.051182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.051907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.052257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.052635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.053391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.053707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.055057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.067766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 403/808] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.068627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.093697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.126454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.134657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fab8a-d0a5-47ed-a44c-374ef6244114 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.167947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.168125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586711cf-436a-4bc1-bb7c-8c41b870817b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.168604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.168886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.169158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.169780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.171140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:36.180501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:36.181406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:36.182255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 406/809] 10.222.0.144 () {56 vars in 1134 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:36.183096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.206749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.207360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.207638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.207907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.208622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.208926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.210403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a2b3-e913-4bd1-8d89-4431552cae1b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.225418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 404/810] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.226695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589aa4-c79b-48b3-8caf-6603465632d0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.226988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589aa4-c79b-48b3-8caf-6603465632d0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.227290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25589aa4-c79b-48b3-8caf-6603465632d0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.228494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 405/811] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:36.229191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25589aa4-c79b-48b3-8caf-6603465632d0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.308639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad6b7-405a-444b-b3b3-1ff6c7032cef None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.320553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.336777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25589aa4-c79b-48b3-8caf-6603465632d0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.342385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.343042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.343325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.343597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.344336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.344665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.346605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.357686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 407/812] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.358462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.370779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.371413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.371697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.371970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.378527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.378947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.379974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:36.391165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:36.392835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 406/813] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:36.393981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.467869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cff54-9290-47a3-95d9-68af57b570e7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.496357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287fe26-2dce-49d2-8850-4b84c56b31d5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.502421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.503037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.503328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.503598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.504279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.504622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.505983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f4237-6689-44fb-ae6b-6cd09c4bfac4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.510716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 408/814] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.511692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.511975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.512246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.529937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:36.531101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:36.531597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.532231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.532575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.533407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.544992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.547561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:36.562405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:36.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:14:36.564219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:36.568248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 407/815] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:36.568928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.572319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.597324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 409/816] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:36 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:36.598157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.609234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.679842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80705db6-9cc8-4013-a858-5dd8674a2dab None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.698198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ecfd2-3fe3-4269-972c-c0c3e4c8179b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.713920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.714575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.714853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.715117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/24f380d935524a2295fd9c01a8fec95b/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.717055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=24f380d935524a2295fd9c01a8fec95b, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.718473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.732151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.732784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.733065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.735038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.735038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.735038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.735733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.747212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 410/817] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.748073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.788950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 408/818] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:36 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/24f380d935524a2295fd9c01a8fec95b/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:36.789766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.829241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.863378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:36.865309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc294b24-31e4-4844-8357-40c72c97d43b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.910757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcefd6b-ae09-4f9f-a9db-ab71267e951c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:36.913916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.914845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.918583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.918583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.918583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.918583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.918583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.930253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 411/819] 10.222.0.144 () {56 vars in 1129 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.930253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:36.961459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:36.962083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:36.962413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:36.962691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:36.963498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:36.963808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:36.965276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:36.984831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 409/820] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:36.986112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.037288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87dd3-661c-4841-adeb-51792e71cf4e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.072025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.072710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.072976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.073248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] PATH_INFO: `/v3/services/network` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.073870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.075223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.077092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:37.090223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:37.091383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:37.092211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 412/821] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:36 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:37.093030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.093099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d02ce-c1ce-4754-a486-2ca26caefa29 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.117904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.130098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.133093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.133093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.133093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/c0f8e4e0592447ef8cc91842386e25bd/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.146623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=c0f8e4e0592447ef8cc91842386e25bd, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.148296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.193860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6461-c172-4c2e-9706-06440d25c448 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.226435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.227046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.227330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.227655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.228368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.229749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.245779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 410/822] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:36 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/c0f8e4e0592447ef8cc91842386e25bd/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:14:37.246704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.337005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 413/823] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:37 2022] POST /identity/v3/endpoints => generated 348 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:37.338457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.400381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54c0b0-6d5d-47fc-bbd1-87b90ff36e54 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.425914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.463091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.463809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.464092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.464368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.495256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383ddbd-a159-4dc6-aea2-995051d4495a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.540774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.561995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.562617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.562948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.564114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.564114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] RBAC: Adding query filter params (name=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:37.564560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.567050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.573281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 414/824] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:37 2022] GET /identity/v3/services?name=network => generated 125 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:37.576817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.578572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.583667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.705069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 411/825] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:37 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:14:37.710076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.710309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55834-3986-43ff-a2f5-96d9939c5a27 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.735124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.746235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.747147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.747433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.747717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.748370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] RBAC: Adding query filter params (type=network) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:37.758071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.758071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.758071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 415/826] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:37 2022] GET /identity/v3/services?type=network => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:37.761910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:37.788420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:37.847300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f35dc3-3581-40b5-a1c6-f6f48d907d9c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.916616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:37.917263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:37.917535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:37.917807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:37.937388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:37.938943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:37.958839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1790185-2348-4650-97be-6978fe87bf79 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:37.977663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:37.978599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:37.979868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:37.979868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:37.979868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:37.979868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:37.979868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 412/827] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:37.986923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:38.001304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:38.001954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.002231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.002500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.003212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:38.004584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:38.099207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 416/828] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:37 2022] POST /identity/v3/endpoints => generated 329 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:38.129401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5dfe8-84c6-4134-85f5-fabc6fbbeb8c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:38.159308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:38.190046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:38.190686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.190964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.191233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.191936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:38.192237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:38.193641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:38.218934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 413/829] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:38.225266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:38.277562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:38.348047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266f32-2edd-434a-b2a1-7f9a822d9137 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:38.383462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:38.384111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.384407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.384692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.391891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:38.393942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:38.405353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:38.406282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:38.407299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 417/830] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:38.412269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:38.531235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ec4b-b67c-4bd7-8328-897bcccf53e6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:38.570756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:38.571374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.571637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.571909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.572701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:38.573009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:38.574431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6aa267-1be5-4b3c-90c7-ca7e06a999e2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:38.588576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 414/831] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:38.597510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:38.712271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:38.716676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fcc6d-f769-4063-b7f5-6b8e1db5f825 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:38.756045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:38.756608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.756890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.757165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.757960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:38.758272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:38.759699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111de06-c82f-448a-9f6c-1e98b6e25ff1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:38.766218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 418/832] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:38 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:38.995011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd651-d198-4637-846c-ac5a2c5a9a60 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:38.995364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd651-d198-4637-846c-ac5a2c5a9a60 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:38.995573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd651-d198-4637-846c-ac5a2c5a9a60 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:38.997171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 415/833] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:39.003077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.003366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.003644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.023453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:39.024592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:39.094914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 419/834] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:39 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:39.117421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dd651-d198-4637-846c-ac5a2c5a9a60 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.155925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:39.224577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dd651-d198-4637-846c-ac5a2c5a9a60 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.259479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.260116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.260414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.260703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.272333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:39.277342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:39.278268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:39.279506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 416/835] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:39.284742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.393587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:39.402294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664eea5-9efd-4d63-947a-b95f79d812f8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.437813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.438435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.438717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.439018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.439741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:39.440044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:39.441437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3120322-bce5-4fab-a72e-d0111ccd4b7e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:39.450419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 420/836] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:39.456596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.562065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb1c7-7fd9-49aa-9095-9529119c8903 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.564751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:39.596684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.597335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.597582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.597851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.608917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:39.610366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:39.626184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:39.627617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:39.628546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:39.628546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:39.628546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:39.628546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:39.628546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 417/837] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:39.629086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bc43-1d3a-47af-9fa1-2d7d8ec62590 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.629256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bc43-1d3a-47af-9fa1-2d7d8ec62590 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.632412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bc43-1d3a-47af-9fa1-2d7d8ec62590 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.632412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 421/838] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:39.633109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.636426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.636725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.637005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.666062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:39.667789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:39.748632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809f7e-4706-4f80-8f5e-8ce9937319cf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.759438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 422/839] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:39 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:39.783204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.800122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.801299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.801790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.802303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.803199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:39.803710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:39.805277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6dac-ba25-4f11-9f5d-c7c8da3c019e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:39.818734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 418/840] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:39.820361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d12b-7635-4ab6-8108-9ff66b05a897 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.820684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d12b-7635-4ab6-8108-9ff66b05a897 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.820959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563d12b-7635-4ab6-8108-9ff66b05a897 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.822168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 419/841] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:39.823792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d12b-7635-4ab6-8108-9ff66b05a897 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:39.835299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:39.880999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:39.897789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ccc61-9381-4d18-9d72-a1716e2eca3b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.936088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563d12b-7635-4ab6-8108-9ff66b05a897 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:39.938072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.938740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.939543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.939808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.952331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e92cc037-6a30-4b93-b5f0-e0a427373062 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:39.957899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:39.958803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:14:39.959727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:39.963872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 423/842] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:39.965238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.965519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.965791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.981219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:39.982143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:39.982628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:39.983078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:39.983443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:39.984655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:39.991857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:39.993948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:40.006413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:40.007513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:40.008621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 420/843] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:14:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:40.009189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.059320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 424/844] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:39 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:40.061642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.083811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.117410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadaab-0c2f-4d92-a65e-60efcc59b84a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.150802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.152062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.152697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.152979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.153255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.153965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:40.154267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.155688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.169115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 421/845] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:40.169115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.187279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf0a62-09fc-4783-87a2-6eabf76f071b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.221215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.222082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.222368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.222642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.223438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:40.223754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.225218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.243499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 425/846] 10.222.0.144 () {56 vars in 1140 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:40.244307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.281472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97ea3-76c7-490c-809f-fc826dc1a017 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.311812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.318537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.319184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.319476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.319745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.320447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.325446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:40.330944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:40.331902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:40.332746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 422/847] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:40.333518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.357804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e80442-96b2-4dfb-809b-0d22652102d0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.395883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.402131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.405649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.405649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.405649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.416400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.417901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:40.433346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:40.434522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:40.435412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:40.435412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:40.435412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:40.435412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:40.435412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 426/848] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:40.435703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.443045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7da05-6eb7-4a60-b541-e4b527028357 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.484573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.485446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.485560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.485821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.551063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7774c-523f-4b1d-8980-67c569801387 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.551266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.559727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.561268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.588282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.589002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.589284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.589564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.590241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:40.590547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.591898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.596538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 427/849] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:40.597310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.649406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.675411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 423/850] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:14:40.676101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.711675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314bd87-6d80-4e18-9ff8-2fcf70e2bc94 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.763006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.763764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.764047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.764463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.765191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:40.765454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.766819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.776231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.783816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 428/851] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:40.783816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.794515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455912d-2308-498d-99ce-c449f33c2b92 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.823791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.824432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.824719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.824996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.825672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:40.825982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.827356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.840415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 424/852] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:40.840415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:40.903277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4a26c-3eba-4d82-8809-8f1699a92b97 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.914843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:40.944534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:40.945187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.945472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.945745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.956461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0133d-9ad2-4019-b7ff-8c74505c874d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:40.956778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:40.958255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-72d1e7aa-68f2-409d-acfa-10a389816066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:40.974186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:40.975337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:40.976288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:40.976288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:40.976288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:40.976288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:40.976288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 429/853] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:40.976638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445d60-45e7-49dd-bad8-9d0c2e9b39ef None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:40.976924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445d60-45e7-49dd-bad8-9d0c2e9b39ef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:40.977197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9445d60-45e7-49dd-bad8-9d0c2e9b39ef None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:40.978627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 430/854] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:40.983204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9445d60-45e7-49dd-bad8-9d0c2e9b39ef None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.004702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.005350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.005625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.005896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.013503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.015161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c78910c6-f85e-48b9-a14c-b58fb646a55e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:41.028361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:41.029603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:41.030547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 425/855] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:41.031148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.031148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.031277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.049378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:41.050552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:41.065286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.106178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9445d60-45e7-49dd-bad8-9d0c2e9b39ef None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.122576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 426/856] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:41 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:41.133356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.150155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.150773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.151052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.151324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.152016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:41.152466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.154001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.168549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 431/857] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:41.169405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.197416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.267191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a0cb9-3294-4fe8-8ff5-e246ed1aea5b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.279376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.279993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.280269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.280610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.281464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:41.281769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.283245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff1586-aca0-4003-9aed-1f3b845b4726 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.283726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.316443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 427/858] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:41.316443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.328154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.337416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.338052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.338338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.344302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.344302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.344302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:41.349988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:41.350900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:41.351705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 432/859] 10.222.0.144 () {56 vars in 1137 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:41.352563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.434704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c15c29-217e-43ef-a6de-bcd0ee9f25d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.449291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.465505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395c38-5e37-477d-a9e7-b14f66508552 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.472121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.472121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.472121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.472121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.480614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.482061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:41.497185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:41.498041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:41.498894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 428/860] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:41.499445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.514767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.515722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.516067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.516440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.572726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.588043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.589762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.608729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781b556-0fb5-42a3-b9ee-27679f918715 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.644924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.645647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.645927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.646199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.646870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:41.647176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.648777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.658689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 429/861] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:41.658689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.681314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.701931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 433/862] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:14:41.701931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.764832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d4d0b-fa19-4d5b-84f7-1bf84689fb19 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.796842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.802952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.803653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.804001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.804348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.805192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:41.805590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.807079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.810679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ccd93-9398-4eb0-95f5-ef7bbae9217a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.818822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 430/863] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:41.834265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.855903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.856562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.856845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.857114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.857792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:41.858095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.859850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:41.876418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 434/864] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:41.876418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:41.926265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3ddd-6a1f-4e29-819d-003fcd1ce3de None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:41.962707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:41.966887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:41.967550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:41.967830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:41.968101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:41.982729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:41.984193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:42.000525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:42.002038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:42.002891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:42.002891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:42.002891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:42.002891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:42.002891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 431/865] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:42.009550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:42.019976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53da1a-0297-4e7c-aeaf-d8b34348bd03 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:42.068812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:42.070975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:42.071497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.072209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.072678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.073809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=24f380d935524a2295fd9c01a8fec95b) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:42.074699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:42.076127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.084657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 435/866] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:41 2022] GET /identity/v3/role_assignments?user.id=24f380d935524a2295fd9c01a8fec95b&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:42.145427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b78168-1ade-42b4-b6c1-1ad2061306b4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:42.184504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:42.185133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.185411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.185680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.186386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:42.186678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:42.188315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.202924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 432/867] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:42.208866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:42.317282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945476d-5688-450b-a665-7a366ae1f2cf None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:42.332864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:42.354643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:42.355392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.355717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.355995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.363748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:42.365486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:42.378760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:42.379666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:42.380573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 436/868] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:42.385768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:42.491695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4765-6b99-48db-a8e3-1d5fc1544f92 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:42.496409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:42.527733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:42.528418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.528747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.529020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.529765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:42.530070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:42.531590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58224d-9e45-4e87-8b4d-264f9ac63657 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.544678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 433/869] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:42.550965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:42.656037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c53c9-4f20-4a79-b135-821f4944f316 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:42.692912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:42.693916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.694404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.694927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.695906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=c0f8e4e0592447ef8cc91842386e25bd) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:42.696449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:42.698064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce71f3-f0d9-4e05-9845-3819fefb4fe5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:42.708743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 437/870] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:42 2022] GET /identity/v3/role_assignments?user.id=c0f8e4e0592447ef8cc91842386e25bd&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:42.898689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5aa9-2d0e-448c-8848-895fd380a3c5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.899198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5aa9-2d0e-448c-8848-895fd380a3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.899300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e5aa9-2d0e-448c-8848-895fd380a3c5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.900960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 434/871] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:42.906176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4c77-0e0c-426b-bbbd-ee04192eb0e8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:42.906634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4c77-0e0c-426b-bbbd-ee04192eb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:42.907056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4c77-0e0c-426b-bbbd-ee04192eb0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:42.927881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d5ed4c77-0e0c-426b-bbbd-ee04192eb0e8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:42.929385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d5ed4c77-0e0c-426b-bbbd-ee04192eb0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:42.964750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:43.003689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 438/872] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:42 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:43.026759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e5aa9-2d0e-448c-8848-895fd380a3c5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:43.136261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e5aa9-2d0e-448c-8848-895fd380a3c5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:43.170735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:43.171324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:43.171610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:43.171935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:43.172670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:43.174462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cbc8a-119a-484f-932b-236c28d14f10 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:43.252251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 435/873] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:43 2022] POST /identity/v3/services => generated 252 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:43.953325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068780-d9b3-4cf9-9a46-d57063218813 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:43.953850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068780-d9b3-4cf9-9a46-d57063218813 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:43.953956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c068780-d9b3-4cf9-9a46-d57063218813 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:43.956015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 439/874] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:43.961375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86689-a23f-49b2-bfe2-7330626d965d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:43.962209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86689-a23f-49b2-bfe2-7330626d965d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:43.962686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f86689-a23f-49b2-bfe2-7330626d965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:43.981733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-36f86689-a23f-49b2-bfe2-7330626d965d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:43.983273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-36f86689-a23f-49b2-bfe2-7330626d965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:44.005610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:44.064482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 436/875] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:43 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:44.086214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c068780-d9b3-4cf9-9a46-d57063218813 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:44.193772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c068780-d9b3-4cf9-9a46-d57063218813 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:44.230611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:44.231151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.231432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.231709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.232427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:44.233802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:44.272363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dfaf6-ea96-4502-a34e-3b6120cde54e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.272699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dfaf6-ea96-4502-a34e-3b6120cde54e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.273003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dfaf6-ea96-4502-a34e-3b6120cde54e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.274482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 437/876] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:44.279584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.279927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.280209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.299683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:44.300966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:44.349687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 440/877] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:44 2022] POST /identity/v3/services => generated 270 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:44.393402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 438/878] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:44 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:44.415686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:44.453096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:44.543011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf296c-cad3-47bd-80da-d9ba4b4847de None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:44.580187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:44.580864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.581145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.581418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.582041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:44.583412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:44.592620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:44.593590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:44.594399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 441/879] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:44.600074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:44.685132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:44.710938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a2ca-0151-4197-92d2-f6018a62cec2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:44.745686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:44.746229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.746510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.746783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.747725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:44.748233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:44.750053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ebd0e-9d5e-4226-b4f4-91df2e51328b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:44.757796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 439/880] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:44 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:44.761464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:44.856528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:44.866573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1799b6-a28a-49d2-9414-5d76ef73a9d5 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:44.878430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c270a-bc8e-46a2-93ed-50f85001ab5f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.878777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c270a-bc8e-46a2-93ed-50f85001ab5f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.879050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c270a-bc8e-46a2-93ed-50f85001ab5f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.880618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 440/881] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:44.886885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.887164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.887436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.906228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:44.907465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:44.914987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:44.916232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:44.916745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:44.917193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:44.918374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:44.919163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:44.920979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:44.927930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 442/882] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:44 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:44.988568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 441/883] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:44 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:45.012195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.121377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4c409-d566-4b9c-b3f1-4190f9d0fe11 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.157039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.158082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.158558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.159024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.169609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:45.175385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:45.176360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:45.177229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:45.179993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 443/884] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:45.190021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.258380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.291893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1540de-9889-4343-99e9-6b0796fe8183 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.327487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.328105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.328406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.328687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.329432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:45.329738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.331098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee6f6-6fea-403f-aca2-4cc165effc09 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.339767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 442/885] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:45.346032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.363369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.438654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132843-8f6e-4b3f-b40e-4aa27b17a90c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.439314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132843-8f6e-4b3f-b40e-4aa27b17a90c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.439795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132843-8f6e-4b3f-b40e-4aa27b17a90c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.441623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 443/886] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:45.446699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.447211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.447702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.450564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2fccd-29b6-4416-9f6c-4731b4a4b578 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.467177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:45.468759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:45.486910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.487544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.487822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.488310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.499735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.501405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:45.517495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:45.518516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:45.519381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:45.519381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:45.519381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:45.519381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:45.519381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 444/887] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:45.524436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.548540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 444/888] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:45 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:45.566950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.616760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.628717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.632719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef99bff-3846-4ec4-a43c-6fb5df6fa11f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.671651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9904423-ff74-45de-9c88-dfa513be219e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.676975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.677919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.678569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.679045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.680404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:45.680892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.682905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.702197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 445/889] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:45.707850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.712195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.713061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.713341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.713614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.714241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.715599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:45.726356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:45.729169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:45.730296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 445/890] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:45.733372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.833409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95839ec-ae33-42d8-b2db-ad542f3a4571 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.852922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbf0c5-14f2-4fb8-8ece-429d98035b93 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:45.867690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.877569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.882243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.882243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.882243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.883450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:45.887153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.888831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.895936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:45.896830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:45.897308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:45.897842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:45.898703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:45.899218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:45.900818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:45.903909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:45.904840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:45.905725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:45.908573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 446/891] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:45.911259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:45.911490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 446/892] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:45.918947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.029006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44154be3-f4ff-4a8b-b243-24121789bf7a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.049634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89a96-8252-454b-81c9-5d950ce9660b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.076281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.077455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.077883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.078300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.079804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:46.080286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.082340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.095498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 447/893] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:46.096040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.097507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.098390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.099199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.100732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.101577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:46.101890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.103344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.122614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 447/894] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:46.128277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.161913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.188093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.224999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce314e-4355-4d35-904e-0ef8d8ae4a34 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.252054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0ac4-647c-459c-8a49-f3fc91e8dca1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.261576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.262511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.263042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.263551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.264544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] RBAC: Adding query filter params (interface=public, service_id=ab3bdb8ca9a34c008e31af807f56405b, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:46.265076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.266674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.272374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 448/895] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/endpoints?service_id=ab3bdb8ca9a34c008e31af807f56405b&interface=public®ion_id=RegionOne => generated 195 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:46.277652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.296034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.296997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.297487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.298498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.380419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.381962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.402807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d35f6-aa63-460d-afaf-b55f5a7242a1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.420788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.439815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.440857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.441347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.441831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.442693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.444277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f717f5-bb5c-49ed-8ea4-adfed7f7aa66 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.448756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.451680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 449/896] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/services => generated 1274 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:14:46.509290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd500a-c483-4832-a6db-806b473989ad None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.509816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd500a-c483-4832-a6db-806b473989ad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.510237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd500a-c483-4832-a6db-806b473989ad None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.512096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 450/897] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:46 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:46.518641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.519283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.519910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.527662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 448/898] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/auth/tokens => generated 1364 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:14:46.533956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.542345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:46.544040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:46.622963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 451/899] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:46 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:46.646599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.662735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c137a04-c45e-46bd-ae1d-e095e62bdbc0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.707203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.707850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.708131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.708433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.719882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.721460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.724514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:46.738346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:46.739284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:46.740163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:46.740163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:46.740163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:46.740163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:46.740163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 449/900] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:46.745368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.759511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e97bd-08a1-4ec9-9db8-07ec16d07b3f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.771841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.796217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.797179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.797670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.798155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.798999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.800583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:46.810083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:46.811015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:46.811839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:46.817734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 452/901] 10.222.0.144 () {56 vars in 1137 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:46.825111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.858335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd05e5-c76e-495d-bf43-ad828d4c968b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.895554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.896450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.896934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.897359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.898260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:46.898801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.900343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.917815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 450/902] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:46.919013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:46.937476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3347-9062-4467-9bfd-b2d117f8f6ab None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:46.975414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:46.976153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.976626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.977127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.978007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:46.978502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:46.980231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:46.981020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9cce4-c340-4e73-b6b7-371dc7321e81 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:46.988435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 453/903] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:46.994393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ddf8-34a7-44ec-a5ba-bccbbd15a68e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:46.999541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ddf8-34a7-44ec-a5ba-bccbbd15a68e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:46.999541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ddf8-34a7-44ec-a5ba-bccbbd15a68e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:46.999541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 454/904] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:46.999541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9ddf8-34a7-44ec-a5ba-bccbbd15a68e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.027248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca2f4-6273-457e-bef2-9b6d9c51c5c7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.038610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:47.070638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.071279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.071561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.071834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.078735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.080200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bc3cecc1-0ea6-4701-b61c-3e01671be800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:47.092795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:47.093930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:47.095153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:47.098837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 451/905] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:47.100123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.100611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.100984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.109778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9ddf8-34a7-44ec-a5ba-bccbbd15a68e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.120022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:47.121254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:47.147443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.148133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.148434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.148713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.149381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:47.149683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.151075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.161080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 455/906] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:46 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:47.161080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.196568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 452/907] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:47 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:47.197256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.255793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:47.268934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5a97f-2436-4d29-b9a5-4f0d8c26e155 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.270593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:47.303830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.307394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.307394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.307394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.307394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:47.307394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.307851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.320804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 456/908] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:47.321828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.322258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5510e17-57f1-4172-a67b-b4c834147cfe None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.365736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.366364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.366651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.366926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.367764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] RBAC: Adding query filter params (interface=public, service_id=7a7904b6f4eb4483bca1cfc0aad1aea2, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:47.368078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.369498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.374962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 453/909] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:47 2022] GET /identity/v3/endpoints?service_id=7a7904b6f4eb4483bca1cfc0aad1aea2&interface=public®ion_id=RegionOne => generated 195 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:47.375752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.440939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43751a6-1434-48e2-b52a-23f4d699f357 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.482220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.482845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.483141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.483423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.484084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.486037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bddc2-0637-4d24-9629-ef569e000eee None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.487278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5134f-4c87-4fcc-a94d-5eb281c77afd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.498448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:47.522079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.522691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.522975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.523250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.524038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] RBAC: Adding query filter params (role.id=95b534da3fb4459096d20252e299be11, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=c0f8e4e0592447ef8cc91842386e25bd) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:47.524353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.525798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.531199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 454/910] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:47 2022] GET /identity/v3/role_assignments?user.id=c0f8e4e0592447ef8cc91842386e25bd&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=95b534da3fb4459096d20252e299be11 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:47.531800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:47.554838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 457/911] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:47 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:47.660256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393eb08-fac4-44dc-b28b-5b81c4e8f0f2 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:47.717637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:47.718279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:47.718566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:47.718841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:47.719694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:47.721145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6939a6-4104-4a44-a794-c9e298698091 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:47.728760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 455/912] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:47 2022] GET /identity/v3/services => generated 1504 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:14:48.579832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6972ac-3531-4f9d-9d0a-043716f9a65b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:48.580375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6972ac-3531-4f9d-9d0a-043716f9a65b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:48.580375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6972ac-3531-4f9d-9d0a-043716f9a65b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:48.581750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 458/913] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:48.587083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:48.587469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:48.587710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:48.606876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:48.608066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:48.635873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:48.678782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 456/914] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:48 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:48.701390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6972ac-3531-4f9d-9d0a-043716f9a65b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:48.730485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:48.813312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6972ac-3531-4f9d-9d0a-043716f9a65b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:48.853387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:48.854015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:48.854297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:48.854572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:48.855200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:48.856588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:48.865996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:48.867087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:48.867950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 459/915] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:48.877569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:48.981650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b504dcb-95e1-41b8-b43e-6adc16a8d3f6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:49.016638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:49.017259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.017545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.017822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.018498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:49.018807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:49.020174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:49.025466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 457/916] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:48 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:49.031183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:49.122674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:49.137746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26824d0d-ba81-4f97-8d54-e5bbab91da86 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:49.173218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:49.173839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.174122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.174396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.175079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:49.175384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:49.176844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db57fcb-937b-4d92-a90c-d842a23fa90e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:49.182157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 460/917] 10.222.0.144 () {56 vars in 1119 bytes} [Tue Jul 26 16:14:49 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:49.187884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:49.220594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:49.301814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64520-0ef9-4f1e-81cb-21eec860e4f1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:49.337366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:49.337982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.338307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.338582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.339303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:49.340679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfa3c9-965c-4f2a-8496-2f34f37e71b8 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:49.418830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 458/918] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:49 2022] POST /identity/v3/endpoints => generated 348 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:49.626503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7605-934f-448b-a674-4e5365fca26e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.627049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7605-934f-448b-a674-4e5365fca26e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.627049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7605-934f-448b-a674-4e5365fca26e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.628699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 461/919] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:49.633879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.634756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.635256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.654211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:49.655749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:49.724358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b9ab-52ae-49f4-a986-9361861035fa None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.724561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b9ab-52ae-49f4-a986-9361861035fa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.724828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b9ab-52ae-49f4-a986-9361861035fa None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.726332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 462/920] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:49.732852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 459/921] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:49 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:49.733017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56593-0c9d-4dc3-bfdc-12debe7f93ce None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.733385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56593-0c9d-4dc3-bfdc-12debe7f93ce None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.733837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56593-0c9d-4dc3-bfdc-12debe7f93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.758209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:49.759312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b7a56593-0c9d-4dc3-bfdc-12debe7f93ce None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:49.760613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7a56593-0c9d-4dc3-bfdc-12debe7f93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:49.772530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:49.848543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 463/922] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:49 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:49.849871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf0af-de3b-4d63-a36d-db7f9b906900 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.850155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf0af-de3b-4d63-a36d-db7f9b906900 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.850432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcf0af-de3b-4d63-a36d-db7f9b906900 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.852544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 464/923] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:49.857627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.857937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.858188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.878556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3d24f-d15c-4cad-9bd4-5c4cfbd13bc2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:49.879848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:49.881084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:49.922308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:49.923398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:49.923685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:49.923955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:49.935167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:49.940128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:49.941104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 460/924] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:14:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:49.942646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:49.960615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 465/925] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:49 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:49.960615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:49.979586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.027836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.062711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd270d-7eec-4699-9410-fd0ed6f5a73c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.075335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52aea9-44d8-48cf-a863-5aa7c1070bf4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.101594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.102343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.102624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.102900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.103532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.105156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.109499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.110153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.110441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.110712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.111418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.111720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.113094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:50.114415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:50.115334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.116308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 461/926] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:50.117111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.131567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 466/927] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:14:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.131567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.207305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.230014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198216-f250-4bbd-8b2b-c037032e40ac None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.239299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b685fe-e670-4b55-b506-0655582b058d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.269349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.269988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.270271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.270741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.271165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.272558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.274406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.275939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.276580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.276859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.277118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.277782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.278080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.279413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:50.282261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:50.287494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.288297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 462/928] 10.222.0.144 () {56 vars in 1137 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:50.289167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.289751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 467/929] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.290596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.394318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c24a-fdf5-4a54-8406-50c4729b6385 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.395392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370197be-e845-4965-94e0-c33e848368d8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.428559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.429190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.429488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.429589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.429771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.430098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.430375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.430647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.431310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.431624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.433034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.435352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.439765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 468/930] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.442246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.443671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.445197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:50.459227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:50.460145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:50.461101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:50.461101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:50.461101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:50.461101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:50.461101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 463/931] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:50.461945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.517545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.565398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e208df-57cc-4a53-acbf-8ed73ed7ef78 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.570925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94662ba-ae1e-48fd-a7b8-1ff4c2b95132 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.607157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.607397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.607777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.607893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.608056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.608150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.608324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.608517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.609025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.609127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.609328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.609422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.611219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.611324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.621077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 464/932] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.621077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.621303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 469/933] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.622532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.687552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.727949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7b710-7c4f-418c-99eb-277139c33fd9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.728908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d491d-022e-4850-82a8-7477246d1a37 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.746032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:50.762775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.763387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.763669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.763936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.764655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.766008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.766356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.766965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.767243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.767513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.768197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.768533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.769861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.782315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 465/934] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.783078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.863661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 470/935] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:50.865087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:50.897897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30d2a-54b6-4b5e-907d-38a6acde2d84 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:50.960255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:50.961256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:50.961682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:50.962132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:50.963453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] RBAC: Adding query filter params (interface=public, service_id=cbecb405470a4648b7b4c462f73e2b31, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:50.964591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:50.966488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:50.977742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 466/936] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/endpoints?service_id=cbecb405470a4648b7b4c462f73e2b31&interface=public®ion_id=RegionOne => generated 195 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:50.983424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.017552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ef71-c521-461c-ac8c-cbc8b074462f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.029836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.042521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.058014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.059196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.059473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.059741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.071404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.074501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:51.086883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:51.087867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:51.089821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 471/937] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:51.096037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.120719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f0c39-2189-4077-891e-26c28e00587f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.159437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.160059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.160334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.160633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.161392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.162928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.176563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 467/938] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:50 2022] GET /identity/v3/services => generated 1504 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:14:51.215583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2c818-1d9f-4533-8e1f-b8ce391d2b98 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.255200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.255839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.256122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.256421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.257242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:51.257551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.259055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.279995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 472/939] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:51.286265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.393208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.397744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78097a-30e6-4956-b84a-cd3aaaf27514 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.437355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.438554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.439098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.439658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.450900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.452647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bd522eab-4bcc-46ae-82f0-a2b37fb683c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:51.468489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:51.469454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:51.470878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:51.470878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:51.470878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:51.470878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.473185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 468/940] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:51.480701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.506443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383085b-beb1-49a1-8487-37b0cdbc7ff2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.506733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383085b-beb1-49a1-8487-37b0cdbc7ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.507023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383085b-beb1-49a1-8487-37b0cdbc7ff2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.508650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 469/941] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:51.513436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.513725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.514006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.531183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.536116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:51.537664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:51.595466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70485cf8-709c-4242-8585-f43b691df5d6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.626053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 470/942] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:51 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:51.635438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.636311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.636877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.637364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.638250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:51.638784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.640401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.650185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.655033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 473/943] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:51.662800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.743595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.759684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30368046-71e0-438e-84c8-d32b3f057499 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.769820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096e5f3-8caf-4c9b-bec0-2bb4df1851c5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.795434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.795924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.796207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.796551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.797247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.799394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.802002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING keystone.server.flask.common [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 16:14:51.807272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:51.808126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:51.808875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:51.809326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:51.816335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:51.818165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:51.818796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:51.830744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:51.831592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:51.832511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:51.838916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 474/944] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:51.839862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.851196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.sql.core [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:14:51.851196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 16:14:51.851196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [parameters: {'id': '51279fb298f441c19ab4b29e6d96592a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 16:14:51.851196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80369) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51279fb298f441c19ab4b29e6d96592a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51279fb298f441c19ab4b29e6d96592a. Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:51.858453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 16:14:51.869062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application yield resource Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise self.value Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application session.commit() Jul 26 16:14:51.869982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:14:51.870854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:14:51.871755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:51.873991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [parameters: {'id': '51279fb298f441c19ab4b29e6d96592a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.876966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:51.878871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 51279fb298f441c19ab4b29e6d96592a. Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 471/945] 10.222.0.144 () {60 vars in 1145 bytes} [Tue Jul 26 16:14:51 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:51.881436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:51.973200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fd588-db3f-4720-b413-bc9d6ac4ec0e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:51.985662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498084f-b97f-4b14-bbf4-0e258d2d790d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:52.013412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:52.014528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:52.015017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:52.015497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:52.016618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:52.017233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:52.018875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:52.025562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:52.026221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:52.026501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:52.026779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:52.034272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:52.036030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:52.037729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:52.038908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 475/946] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:52.044622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:52.056830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:52.057724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:52.058894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 472/947] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:52.067753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:52.082676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:52.177990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba56a3-59d9-4c67-8186-c6516cb8bd9e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:52.215294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:52.215686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5029e-2d66-4a94-97f3-41e9ffd74201 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:52.215953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:52.216235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:52.216543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/c0f8e4e0592447ef8cc91842386e25bd/roles/95b534da3fb4459096d20252e299be11` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:52.218885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=c0f8e4e0592447ef8cc91842386e25bd, role_id=95b534da3fb4459096d20252e299be11)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:52.220413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c87f3-c880-4bfb-8123-cad2a6cb3245 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:52.253582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:52.254481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:52.254966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:52.255420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:52.256310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:52.256791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:52.258351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc25e2-1122-471f-a669-f7035d4fcaf2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:52.283931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 473/948] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:52 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:52.293043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 476/949] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:14:52 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/c0f8e4e0592447ef8cc91842386e25bd/roles/95b534da3fb4459096d20252e299be11 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:14:53.013545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22892e73-e5e8-4409-bf37-97c1124804bd None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.014069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22892e73-e5e8-4409-bf37-97c1124804bd None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.014069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22892e73-e5e8-4409-bf37-97c1124804bd None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.015803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 474/950] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:53.021747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.022378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.022378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.040929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:53.042112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:53.114717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 477/951] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:53 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:53.132878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22892e73-e5e8-4409-bf37-97c1124804bd None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.194549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.240667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22892e73-e5e8-4409-bf37-97c1124804bd None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.275948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.276550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.276832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.277106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.277732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.279093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-954a37da-5803-46c0-920d-119479f9ca9a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:53.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:53.290114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.290978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 475/952] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:53.296499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.312742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.319985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb32ec6-0e1c-43f9-a90e-1007b894710d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.320269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb32ec6-0e1c-43f9-a90e-1007b894710d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.320580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb32ec6-0e1c-43f9-a90e-1007b894710d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.322175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 476/953] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:53.327007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.327287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.327561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.349137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:53.351250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:53.410660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2229-6ec4-4135-a41d-e8139622791c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.448431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 477/954] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:53 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:53.453645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.455060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.455571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.455682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.456324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:53.456672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.458036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.466768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 478/955] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:53.467986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.469850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.536836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.554592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.585372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6833e-f8b3-426d-939e-1dbf0297288d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.591081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2afbc-866c-410e-982b-921019efd4dc None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.623091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.623751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.624034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.624311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.625053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:53.625321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.627259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.632232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 479/956] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:53.635496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.636381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.636888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.637330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.638145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.639818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:53.650556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:53.651619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:53.652567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:53.655677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 478/957] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:53.665556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.775743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f58f53-ee98-4202-b6c9-f4bda0d4a52e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.811879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.812831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.813900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.814174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.814848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:53.815147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.816619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.818734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.822604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 480/958] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:53.828322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:53.934206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f4136-f07d-4612-95c3-eeb99809c2be None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:53.948911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:53.976068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:53.976713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:53.976995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:53.977269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:53.977951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:53.978254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:53.979630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65aa7-ca00-4d2a-ab26-9b8927f4acbe None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:53.985698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 479/959] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:53 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:53.991175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:54.100283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4907-e99c-45d0-97e7-5c1814739f97 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:54.138421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:54.139040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.139316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.139582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.140300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:54.142279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a7685-6981-40c7-bf1c-00812f06b5e4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:54.235820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 481/960] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:53 2022] POST /identity/v3/endpoints => generated 351 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:14:54.461307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7044-cf22-49a2-b84b-762d3145d76f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.461833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7044-cf22-49a2-b84b-762d3145d76f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.461909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c7044-cf22-49a2-b84b-762d3145d76f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.463524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 480/961] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:54.474500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.474804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.475090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.492913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e1b3-3d80-45be-a421-7c5d0228ea42 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.493193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e1b3-3d80-45be-a421-7c5d0228ea42 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.493469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7e1b3-3d80-45be-a421-7c5d0228ea42 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.495109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 481/962] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:54.500056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:54.500193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.500430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.500752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.501332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:54.521995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:54.523196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:54.560025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:54.591184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 482/963] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:54 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:54.606221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 482/964] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:54 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:54.619595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:54.631223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:54.734716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d80b97-c26e-466e-8221-f8183f1aba9c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:54.759060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975b952-1968-4bf7-9182-86a11162df33 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:54.770807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:54.772243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:54.773012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.773368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.773714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.784958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:14:54.790177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:54.791140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:54.792029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 483/965] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:14:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:54.801400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:54.803415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:54.804102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.804410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.804692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.807071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:54.807071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:54.858316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:54.863324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.sql.core [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:14:54.863324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 16:14:54.863324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [parameters: {'user_id': 'ee40664a0d0e45efb71540981b59f9e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 16:14:54.863324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80369) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:54.872347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:54.878170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application yield resource Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise self.value Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application session.commit() Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:14:54.879115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:14:54.879983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:14:54.880891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application result.read() Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:14:54.881726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee40664a0d0e45efb71540981b59f9e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:54.882554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 16:14:54.883399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:54.884287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 483/966] 10.222.0.144 () {60 vars in 1136 bytes} [Tue Jul 26 16:14:54 2022] POST /identity/v3/users => generated 160 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:54.888982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:54.933845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56f8ae-f6ff-40c4-8518-476368fc8085 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:54.981974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:54.982593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:54.982874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:54.983149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:54.983862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:54.984164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:54.986340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.001635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 484/967] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:14:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:55.003173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.018201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e50e6-9751-4e84-8f64-e8fef585a69c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.063631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.064322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.064632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.064903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.074201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.076308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:55.092444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:55.093379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:55.099208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 484/968] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:14:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:55.101296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.127314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f3a2d-0c46-46c6-9c1c-9f1ba8f4d78b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.129901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.138320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.163700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.164273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.164587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.164747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.178829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.180369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:55.197303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:55.198235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:55.199119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:55.199119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:55.199119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:55.199119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:55.203218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 485/969] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:55.204311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.231622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36d3f-a93a-4dda-ada0-3faadb330415 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.269059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.270021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.270523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.271009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.272006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:55.272619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.274264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.314255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c04aa-1166-4b1c-ac63-7162ae115e41 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.320652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 485/970] 10.222.0.144 () {56 vars in 1106 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:55.355598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.356224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.356533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.356805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.357501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:55.357803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.359295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.374255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 486/971] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:55.380611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.497115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.508257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee26787-d602-4f67-ade2-759d68361908 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.543484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.544502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.544991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.545516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.553472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.555168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8edc6980-24bf-440b-ac57-e8349c16d662 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:14:55.568411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:55.569360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:14:55.570279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:55.571558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 486/972] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:55.577051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.601059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.686688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9e97-7c2e-472a-bdc1-7943dc4f678d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.721017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.721631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.721907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.722173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.722956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:55.723258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.724693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.745113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 487/973] 10.222.0.144 () {56 vars in 1154 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:55.745319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c4f4-2230-41a8-bb1b-1dd4d5567a6b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.745319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c4f4-2230-41a8-bb1b-1dd4d5567a6b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.745319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c4f4-2230-41a8-bb1b-1dd4d5567a6b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.745319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 487/974] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:55.747844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.749843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.750203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.751232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.769860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:55.771511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:55.823429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.849535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 488/975] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:55 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:55.849976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5767089-1f18-48ce-986c-cf06a57ac4d1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:55.871364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:55.887134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:55.887753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:55.888029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:55.888298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:55.958024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:55.959554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:55.963398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:55.989977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289133f-5e11-425b-a551-8522c41c9425 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.029356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.030100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.030381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.030652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.031344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.033295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.034978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.060109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 488/976] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:14:55 2022] GET /identity/v3/auth/tokens => generated 1466 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:14:56.065651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.088513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 489/977] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:14:55 2022] POST /identity/v3/services => generated 243 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:14:56.188993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcbd84-9373-4316-85c7-585a39867bf6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.238849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.239492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.239775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.240039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.251593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.253080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5090c3e3-4022-49ff-88b9-2bfd27962eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:14:56.268619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:56.269537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:56.270444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:14:56.270444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:14:56.270444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:14:56.270444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:56.270444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 489/978] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:56.275437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.350794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74a00-8fc4-4477-92b6-493a31674130 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.351085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74a00-8fc4-4477-92b6-493a31674130 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.351277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74a00-8fc4-4477-92b6-493a31674130 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.352777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 490/979] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:56.357505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.357792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.358071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.376792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:56.377997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:56.382476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f82ec9-851c-4f62-949f-6fd6ac6062e9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.397615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.420345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.420993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.421323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.421597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.422744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:56.423082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.424887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.434907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.441327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 490/980] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:56.446871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.457029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 491/981] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:56 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:56.478859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.553797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12112-33eb-4039-88f2-2004fe782b83 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.592795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb17f0-5b86-442e-bb5e-15b23c013236 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.596779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.597427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.597708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.597981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.605298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.606663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:14:56.619052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:56.619954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:56.621440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 491/982] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:56.626512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.634004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.634653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.634934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.635635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.636268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.638026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:56.647861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:56.649070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:56.649889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 492/983] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:56.654982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.674476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.691948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.736111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927392b-c5d9-4067-aba7-be948d034d7a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.761933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6c21-b68d-4c42-8799-5845e37109ba None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.773823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.774828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.775705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.776175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.777415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:56.778059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.779955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.795174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.795741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 492/984] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:56.795864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.796057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.796333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.797045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:56.797896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.799269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.802499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.809820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 493/985] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:56.811205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:56.897378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.909186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6e417-e1f1-4b2c-a74c-53e890669f99 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.916632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce5c2-5922-41bf-9519-77b286f001dc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:56.926794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:56.951778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.952449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.954017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.954763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.956086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] RBAC: Adding query filter params (role.id=95b534da3fb4459096d20252e299be11, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=c0f8e4e0592447ef8cc91842386e25bd) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:56.956423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.957961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.959446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:56.960242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:56.960574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:56.961314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:56.962001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:56.962314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:56.963669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:56.964226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 493/986] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/role_assignments?user.id=c0f8e4e0592447ef8cc91842386e25bd&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=95b534da3fb4459096d20252e299be11 => generated 610 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:56.974038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 494/987] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:56.975785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:57.088506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b761c6-4b0b-4a2f-9b1d-ee550c3dde94 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:57.131643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:57.132273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:57.132592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:57.132871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:57.133666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] RBAC: Adding query filter params (interface=admin, service_id=cbecb405470a4648b7b4c462f73e2b31, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:57.133962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:57.135335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd8cc7-c98b-44da-861d-bedfa280e6d4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:57.140878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 494/988] 10.222.0.144 () {56 vars in 1254 bytes} [Tue Jul 26 16:14:56 2022] GET /identity/v3/endpoints?service_id=cbecb405470a4648b7b4c462f73e2b31&interface=admin®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:57.147215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:57.255043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017dfc-da0c-4f46-968f-570a69c7094b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:57.288705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:57.289322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:57.289607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:57.289880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:57.290526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:57.291907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ef8cc-a2cd-47b2-883e-a8016ab055be None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:57.298680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 495/989] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:57 2022] GET /identity/v3/services => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:14:58.130408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771550-0939-4541-8258-63491a10d5af None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.131051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771550-0939-4541-8258-63491a10d5af None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.131051 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771550-0939-4541-8258-63491a10d5af None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.132353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 495/990] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:58.137421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.137782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.138160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.157380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:58.158620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:58.160778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:58.231433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 496/991] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:58 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:14:58.253676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771550-0939-4541-8258-63491a10d5af None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:58.302359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:58.357422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771550-0939-4541-8258-63491a10d5af None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:58.392799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:58.393375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.393655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.393929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.394569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:58.396027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:58.406038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:58.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:14:58.409046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:58.410220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 496/992] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:14:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:58.414015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:58.514953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c17b-c4cd-415b-85a3-aec2f40c109e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:58.548286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:58.548930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.549212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.549485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.550155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:58.550460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:58.551826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:58.556766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 497/993] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:58 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:58.562483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:58.672656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3a37e-1611-4c7a-a985-add80bf64eb1 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:58.675500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:58.699483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:58.700103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.700465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.700746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.701419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] RBAC: Adding query filter params (type=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:58.701724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:58.703078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:58.708847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 497/994] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:14:58 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:58.714488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:58.717396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:58.817782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f790291-2ddd-4e4f-a5b5-ba2ea7bdd82c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:58.851961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:58.852605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:58.852889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:58.853167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:58.853916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] RBAC: Adding query filter params (interface=public, service_id=6d4a14d1524d4e2cabb5feb9b01d7c2b, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:58.854223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:58.855586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fc1cb-8078-44f7-9b8a-6d403ef585fc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:58.865833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 498/995] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:14:58 2022] GET /identity/v3/endpoints?service_id=6d4a14d1524d4e2cabb5feb9b01d7c2b&interface=public®ion_id=RegionOne => generated 195 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:58.867129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:58.970965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb0c82-bda8-47f4-8d45-809cacf31fac None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.009816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.010432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.010712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.010984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.011632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.013032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.016066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f5d4-b65a-4548-8aea-85d738ec0f93 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.016352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f5d4-b65a-4548-8aea-85d738ec0f93 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.017511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f5d4-b65a-4548-8aea-85d738ec0f93 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.018119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 499/996] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:14:59.022817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154c1c-a362-41c4-b297-f8ac99c7e58b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.023106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154c1c-a362-41c4-b297-f8ac99c7e58b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.023383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154c1c-a362-41c4-b297-f8ac99c7e58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.026417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 498/997] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:14:58 2022] GET /identity/v3/services => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:14:59.043277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-07154c1c-a362-41c4-b297-f8ac99c7e58b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:59.044524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-07154c1c-a362-41c4-b297-f8ac99c7e58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:59.112851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:59.121482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 500/998] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:59 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:59.144366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.191939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:59.254129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefb0-856a-4f03-b67a-7bf9c58aa0ad None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.288914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.289818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.290248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.290665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] PATH_INFO: `/v3/services/image` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.291456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.292988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:59.302510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:59.303705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:14:59.304737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:14:59.307161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 499/999] 10.222.0.144 () {56 vars in 1110 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:14:59.312880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a90e5f-f2ba-470e-b443-4f5f44cbd79d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.313170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a90e5f-f2ba-470e-b443-4f5f44cbd79d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.313373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.313476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a90e5f-f2ba-470e-b443-4f5f44cbd79d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.318356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 501/1000] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:14:59 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:14:59.322832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.323118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.323391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.341703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:14:59.342903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:14:59.420222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 502/1001] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:14:59 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:14:59.429411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff0619-971b-4a91-a248-8c5e41c0d598 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.431120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:59.443431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.465864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.466728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.467272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.467550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.468346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] RBAC: Adding query filter params (name=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:59.472143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.472143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.480763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 500/1002] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services?name=image => generated 123 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:59.481439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.560500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932d72-88f8-4eb2-989b-4d3fa5f016fd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.589924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:59.591943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b89b4-b16f-47e1-8cb5-21a608d05488 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.599176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.600113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.600643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.601138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.601962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.603450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:14:59.612874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:14:59.614066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:14:59.615110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 503/1003] 10.222.0.144 () {56 vars in 1131 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:14:59.621602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.635299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.635910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.636194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.636491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.637273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] RBAC: Adding query filter params (type=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:59.637580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.638915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.644015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 501/1004] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services?type=image => generated 123 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:59.741742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d725f-a8c9-4527-be26-b7078a35c9f6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.777437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.778067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.778349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.778620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.779308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:59.779611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.781031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.792688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 504/1005] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:14:59.794351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.827182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:14:59.896074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffd68-b9e3-41c0-9c83-243463f7f379 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:14:59.929512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:14:59.930177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:14:59.930464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:14:59.930739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:14:59.931414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:14:59.931718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:14:59.933104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595160d1-bf27-4221-b907-7bd36d2b0066 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:14:59.942903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 502/1006] 10.222.0.144 () {56 vars in 1127 bytes} [Tue Jul 26 16:14:59 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:14:59.944015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:14:59.999915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:00.044945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86524ecc-ec6a-46f8-8429-395eb11a2843 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:00.079115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:00.079645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:00.079924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:00.080249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:00.080982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:00.082403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66265bc-121f-471b-89cb-2da5ce58f0cd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:00.160187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 505/1007] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:14:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:01.066701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c346e-5443-4868-926d-f568eb4420f5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.067226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c346e-5443-4868-926d-f568eb4420f5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.067226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c346e-5443-4868-926d-f568eb4420f5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.068900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 503/1008] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:01.073738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.074117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.074290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.092205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:01.093409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:01.166112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 506/1009] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:01 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:01.188370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c346e-5443-4868-926d-f568eb4420f5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:01.284341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:01.292546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c346e-5443-4868-926d-f568eb4420f5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:01.331209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:01.331763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.332042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.332312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.332969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:01.334466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:01.343227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:01.344213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:15:01.345412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:01.346596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 504/1010] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:15:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:01.350306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:01.394085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:01.452726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f5894-c3ab-491a-a195-3f2a88fd620b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:01.495820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:01.496471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.496755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.497029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.497711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] RBAC: Adding query filter params (name=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:01.498013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:01.499434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:01.508923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 507/1011] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:15:01 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:01.510103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:01.616536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059516a-efae-4269-89a1-3605b40d7318 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:01.650239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:01.650859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.651139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.651412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.652092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:01.652413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:01.653790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087b3e8-af7b-49f9-bf55-08326e8d3b54 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:01.663650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 505/1012] 10.222.0.144 () {56 vars in 1117 bytes} [Tue Jul 26 16:15:01 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:01.665433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:01.680217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d250f4-84de-4222-881e-a53079dbdb45 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.680536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d250f4-84de-4222-881e-a53079dbdb45 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.680812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d250f4-84de-4222-881e-a53079dbdb45 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.682145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 506/1013] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:01.687296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.687582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.687886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.705741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:01.707010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:01.725535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:01.772057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad543bd-a301-4011-97fd-029d18642ded None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:01.774561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:01.777496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 507/1014] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:01 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:01.800529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:01.808213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:01.809594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.810216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.810691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.812060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:01.813789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f86b2-3a51-4061-8ab3-7b68038567ff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:01.896602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 508/1015] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:15:01 2022] POST /identity/v3/endpoints => generated 336 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:01.913434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d348bfd-f2fe-4ace-929f-b4ec419c313b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:01.959944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:01.960600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:01.960881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:01.961151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:01.961853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:01.963223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8e1d4-a938-4c1b-9776-8518866e3ef9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:02.064374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 508/1016] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:15:01 2022] POST /identity/v3/services => generated 243 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:15:02.227067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b77208-766d-4f19-8d12-d2754689f916 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:02.227561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b77208-766d-4f19-8d12-d2754689f916 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:02.227561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b77208-766d-4f19-8d12-d2754689f916 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:02.228958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 509/1017] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:02.234366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:02.234846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:02.234943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:02.255649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:02.256920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:02.259026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:02.332893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 509/1018] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:02 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:02.355551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b77208-766d-4f19-8d12-d2754689f916 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:02.461635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b77208-766d-4f19-8d12-d2754689f916 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:02.504986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:02.505651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:02.505931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:02.506206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:02.507345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:02.508799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ed9c-ce25-46ca-82b0-c4764f33acce None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:02.514976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 510/1019] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:02 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:02.516858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:02.616774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e553c3-579f-4b21-a1be-38aa2d0e4262 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:02.651258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:02.651874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:02.652157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:02.652457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:02.653155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:02.654593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f9c8-fead-4d5d-a4d6-613e737936d9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:02.659415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:02.783911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 510/1020] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:15:02 2022] POST /identity/v3/projects => generated 312 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:15:04.079614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9c95-2c95-437e-95a5-8686d78efc70 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.080138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9c95-2c95-437e-95a5-8686d78efc70 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.080245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a9c95-2c95-437e-95a5-8686d78efc70 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.081714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 511/1021] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:04.086619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.087027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.087169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.105807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:04.106989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:04.119107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85235f3-5744-45f0-9451-36b746b11a0b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.119563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85235f3-5744-45f0-9451-36b746b11a0b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.119844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85235f3-5744-45f0-9451-36b746b11a0b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.121476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 512/1022] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:04.127016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.127303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.127591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.145773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:04.147705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:04.173885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.177934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 511/1023] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:04 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:04.201159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.224712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 513/1024] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:04 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:04.242045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.311133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08d901-e9e2-4b96-a2af-3b24d9126a5e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.327676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.346864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.347439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.347721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.347996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.358373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:04.359792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218278-85f4-4c53-bb96-54542c81513e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:04.363747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:04.364793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:04.365628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:04.372428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 512/1025] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:04.374256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.400971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.402472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.403180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.403639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.404623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.407462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:04.419148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:04.420216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:04.421080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 514/1026] 10.222.0.144 () {56 vars in 1110 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:04.426939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.452882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.488123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e118897-867b-4e96-abea-9731faac7e28 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.531610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.532231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.532542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.532817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.533535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:04.533843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.535202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.541402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cd75-24de-4f48-9e6d-87ed4ceb47ad None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.543664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 513/1027] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:04.549146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.582884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.583735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.584190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.584632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.585438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] RBAC: Adding query filter params (name=image) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:04.585896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.587416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.591361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.592861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 515/1028] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/services?name=image => generated 123 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:04.604481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.659807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6265ea-4af0-4574-9f9b-b5aeb51fd0e9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.688530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.695186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.695826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.696106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.696542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.704563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0314-61a5-4ce1-ba9e-7993d2cbd9ef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.708699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.710143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:04.725699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:04.726873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:04.728033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:04.728033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:04.728033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:04.728033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:04.728033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 514/1029] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:04.732357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.747581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.748198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.748533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.748793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.749467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] RBAC: Adding query filter params (type=image) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:04.749772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.751179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.763814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 516/1030] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/services?type=image => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:04.764836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.837238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.846138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee6480-7f06-4278-a3a0-5413d3398228 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.867503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13748a02-4853-42a9-bbc5-eeaf42767646 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:04.880507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.881315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.881596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.881870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.882644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:04.882959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.884319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca665c8a-1c56-4803-84a6-4dbef798ce80 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.896497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 515/1031] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:04.896497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e1bd-6de3-4d94-80e6-4c52b7c123ec None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.896497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e1bd-6de3-4d94-80e6-4c52b7c123ec None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.896497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e1bd-6de3-4d94-80e6-4c52b7c123ec None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.896497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 516/1032] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:04 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:04.901989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129e1bd-6de3-4d94-80e6-4c52b7c123ec None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:04.904564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:04.905289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.905574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.905843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.907202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] RBAC: Adding query filter params (interface=public, service_id=7452797d79864d22957f6c4e9c1ab528, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:04.907514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:04.908924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729e12-d98b-4797-b2d3-a7cdc9bcd530 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:04.918472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 517/1033] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/endpoints?service_id=7452797d79864d22957f6c4e9c1ab528&interface=public®ion_id=RegionOne => generated 195 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:04.918472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:04.918472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:04.918472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:04.933027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:04.934466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:04.943531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:04.999536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 518/1034] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:04 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:05.000498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.007769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129e1bd-6de3-4d94-80e6-4c52b7c123ec None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.042691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.043328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.043614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.043885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.051355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.053010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:15:05.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:05.065657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 517/1035] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:05.066742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.080777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.104516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fdac9-f8c1-404b-91f7-16600b5b9693 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.137190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.137799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.138076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.138345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.138983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.140332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.147210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 519/1036] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:15:04 2022] GET /identity/v3/services => generated 1966 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:15:05.147978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.159252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.179362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624b9fb-746f-455d-9c75-64565c7690d3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.217604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.218259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.218547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.219430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.220732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.222184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.223855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 518/1037] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:05 2022] GET /identity/v3/domains/default => generated 200 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:05.230010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.266844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c9bfd-05b0-4cf3-ac8c-cf93c8e9ea03 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.304226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.304873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.305151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.305422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.306211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:05.306515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.307947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.318629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.327608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 520/1038] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:15:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:05.333888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.341028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453012ee-34f5-4c8d-8504-4f9346f46033 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.380214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.381067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.381361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.381635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.382316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.383822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.409679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.441650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08189d4c-4a60-44d3-9ae1-ef6ae91a13bd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.468969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 519/1039] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:15:05 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:15:05.483002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.483721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.484555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.485569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.566726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.568199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.637742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.677424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 521/1040] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:05 2022] GET /identity/v3/auth/tokens => generated 1963 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 16:15:05.682835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.787975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db71ad-de3c-4408-8b99-f9c019390dbc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.822417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.823056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.823338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.823611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.825504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:05.835050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:05.836526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:05.850600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:05.851532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:05.852500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:05.852500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:05.852500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:05.852500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:05.852500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 520/1041] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:05.858616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:05.961859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f544d-86ef-4f8a-9861-3dab34dae381 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:05.996852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:05.997223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:05.997429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:05.997706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:05.998386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:05.998691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:06.000087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:06.015912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 522/1042] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:06.017108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:06.116145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d9d06-1035-41d4-98cc-cdfc0643dcac None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:06.149668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:06.150302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:06.150583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:06.150856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:06.157608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:06.159034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:15:06.170246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:06.171159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:06.172065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 521/1043] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:06.178261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:06.207048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:06.280203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e21e62-ccf0-4737-a019-103b943ec85b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:06.316544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:06.317158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:06.317436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:06.317705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:06.318441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:06.318744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:06.320158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5462b7-5869-4eca-a59b-cbc47c2627fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:06.337403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 523/1044] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:06.338631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:06.385625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:06.439527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b8243-bba8-490d-87ab-4e1ae3836d37 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:06.477463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:06.478205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:06.478609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:06.478983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:06.479877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] RBAC: Adding query filter params (role.id=95b534da3fb4459096d20252e299be11, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:06.480290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:06.481828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63581fb-40ec-44db-a89e-1263d8a7774c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:06.496503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 522/1045] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:06 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=95b534da3fb4459096d20252e299be11 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:07.190462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad686-39b6-45fc-8f0d-582a51521654 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.191070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad686-39b6-45fc-8f0d-582a51521654 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.191070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ad686-39b6-45fc-8f0d-582a51521654 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.192342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 524/1046] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:07.197857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.198148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.198424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.216675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:07.217843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:07.285892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 523/1047] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:07 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:07.308866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ad686-39b6-45fc-8f0d-582a51521654 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.396141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:07.412640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ad686-39b6-45fc-8f0d-582a51521654 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.447921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.448505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.448784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.449056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.449665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:07.451014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0cc54a5e-2dbf-4d25-97bc-7f2063c3f757 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:07.459852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:07.460873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:15:07.461801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:07.464702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 525/1048] 10.222.0.144 () {56 vars in 1110 bytes} [Tue Jul 26 16:15:07 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:07.466965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.518829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:07.554400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4a0d-2be4-492b-8ba9-e77fcdd3d8ae None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.554737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4a0d-2be4-492b-8ba9-e77fcdd3d8ae None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.554955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd4a0d-2be4-492b-8ba9-e77fcdd3d8ae None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.556429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 526/1049] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:07.561328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.561558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.561828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.579561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:07.579739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c008-0586-4256-9ead-586cbcca91e4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.580767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:07.616686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.617800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.618384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.618716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.619606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] RBAC: Adding query filter params (name=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:07.619967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:07.621443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:07.632589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 524/1050] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:07 2022] GET /identity/v3/services?name=image => generated 123 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:07.633427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.656651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 527/1051] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:07 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:07.679078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.746582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0910a-e28f-468c-acf4-6f7d91c531dd None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.749674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:07.768854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:07.782294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.783229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.783721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.784571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.785250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:07.785557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:07.786915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:07.788556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6314-7da8-4aed-bdcc-bee612dda1c7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.797473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 525/1052] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:07 2022] GET /identity/v3/services?type=image => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:07.798514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.823919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.824622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.824858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.825129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.836247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:07.841756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:07.842656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:07.843470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 528/1053] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:07.849218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:07.905008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1a764-5af5-4878-ac87-10d316c051fa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.944598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.945210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.945542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.945756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.946467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:07.947808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:07.952193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e3fec-ab99-44bb-98a7-49d6c0d9b24a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:07.987167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:07.987778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:07.988062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:07.988329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:07.989075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:07.989377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:07.990811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:08.000763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 529/1054] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:08.006447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.009923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.034612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 526/1055] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:15:07 2022] POST /identity/v3/endpoints => generated 329 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:08.115791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72c0-0a13-45fc-b71d-77282d846bad None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:08.159303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:08.160209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.160606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.160981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] PATH_INFO: `/v3/users/b2c3a753e3d24ca19ca02ec02dd35539` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.175912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c3a753e3d24ca19ca02ec02dd35539)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:08.177558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:08.185426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 530/1056] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/users/b2c3a753e3d24ca19ca02ec02dd35539 => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:08.185575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.215205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.292004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e2e3-7d0b-42dd-958a-cf32efff9f1d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:08.325987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:08.326614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.326980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.327257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.395254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:08.396733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbf518-f5fe-4b00-b3df-a9f1ea2f0760 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:08.420891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.505763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 527/1057] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 16:15:08.507460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.559977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d21e-5920-4e58-8cb7-6ffa09c8c46f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.560169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d21e-5920-4e58-8cb7-6ffa09c8c46f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.560466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d21e-5920-4e58-8cb7-6ffa09c8c46f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.561976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 528/1058] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:08.566628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.568267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.568629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.568855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.588516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:08.589710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:08.613362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e5a3-29aa-4a55-878a-d767612b71b1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:08.648111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:08.648837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.649116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.649387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.650537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:08.651946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:08.658487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 531/1059] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:08.659812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.663831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 529/1060] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:08 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:08.686882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.745589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.765599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ccf5-b7f7-490a-8809-6773254b8f1d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:08.788739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:08.793559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3ed7e-09e5-4988-85ec-ef47f0e4e733 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:08.799770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:08.800378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.800680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.800951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.801726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f, user.id=b2c3a753e3d24ca19ca02ec02dd35539) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:08.802033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:08.803516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:08.809402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 532/1061] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/role_assignments?user.id=b2c3a753e3d24ca19ca02ec02dd35539&scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:08.832466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:08.833037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:08.833318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:08.833587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:08.845639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:08.850814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:08.851770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:08.852935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:08.856799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 530/1062] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:08.858476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:08.970940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2184-923c-49d9-bd83-70a61f973e22 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.005262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.005871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.006149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.006418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.007118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:09.007419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.008805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.021477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 533/1063] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:15:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:09.023092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:09.157664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dd03-c612-4b35-a9bb-d738cbd2f022 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.172520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:09.193066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.193730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.194012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.194286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.205875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.207341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:09.223742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:09.224687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:09.225557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:09.225557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:09.225557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:09.225557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:09.228447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 531/1064] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:09.230997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:09.289038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:09.338614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41732-d72b-4efe-928c-3defe963e9a9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.372142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.372778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.373057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.373327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.373996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:09.374297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.376683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.389054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 534/1065] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:09.394989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:09.503633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d817-0a26-49b8-b7a1-f75dce42a433 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.538885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.539510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.539781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.540050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.547928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.549431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:15:09.561326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:09.562246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:09.568049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 532/1066] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:09.569143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:09.670411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7104-d740-4792-a036-6891784c3bbc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.681559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:09.704732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.705348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.705627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.705909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.706691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:09.706990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.708420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.731359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 535/1067] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:09.733016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:09.761981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:09.837844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261d56e-880d-44ad-b139-691bf0d42bc3 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:09.874810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:09.875448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:09.875725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:09.875995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:09.886642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:09.888099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:09.902493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:09.903507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:09.904450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:09.904450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:09.904450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:09.904450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:09.908613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 533/1068] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:09.910510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.011086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4ffd-2a11-432a-bb63-ca5028ffb304 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.044471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.045005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.045285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.045560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.046243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:10.046564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.048497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e0d7d-2dff-46eb-bab1-11a0c0bd67de None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.064347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 536/1069] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:10.066186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.103993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d35bb4-6071-4632-8c60-5b77c6128b25 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.104282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d35bb4-6071-4632-8c60-5b77c6128b25 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.104591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d35bb4-6071-4632-8c60-5b77c6128b25 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.106103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 537/1070] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:10.111365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.111651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.111929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.130593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:10.131997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:10.155877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.156814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.178819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577387e6-caa5-4be0-aa8e-d71bb6df2a29 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.203856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 538/1071] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:10.214804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.215449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.215736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.216014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.223264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.224822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.225511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:15:10.237182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:10.238118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:10.239010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 534/1072] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:10.243951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.340971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f5a3b-2caa-47a4-b43b-d3d527f1cb29 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.351290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6281f27-d50d-4f12-811d-f7b946b69211 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.381313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.381927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.382208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.382486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.387427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.388037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.388323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.388624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.389370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:10.389676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.391182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.398515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.403895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 535/1073] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:10.410079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.411900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.457955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.459685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.524691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3724b3-62e0-4aec-8f29-444b3cef6283 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.563408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.564109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.564362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.564646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/95b534da3fb4459096d20252e299be11` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.566996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=6268372f3956497aae05aec64d41e848, role_id=95b534da3fb4459096d20252e299be11)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.569035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.574908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 539/1074] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 16:15:10.580824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.627454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 536/1075] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:15:10 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/6268372f3956497aae05aec64d41e848/roles/95b534da3fb4459096d20252e299be11 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:15:10.649193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.696113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053926e-64bb-4186-8c43-daa31d5210e1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.733094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.733742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.734045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.734848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.746239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.747716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6c5d90ee-9b60-4eaf-be2a-93da7f1075f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:10.763762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:10.764754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:10.773145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:10.773145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:10.773145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:10.773145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:10.773145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 540/1076] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:10.773619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.815806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004964a-7183-4bb2-9d74-b4d49737ec6f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.816151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004964a-7183-4bb2-9d74-b4d49737ec6f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.816352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004964a-7183-4bb2-9d74-b4d49737ec6f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.817946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 541/1077] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:10.822812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.823094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.823378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.841087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:10.841266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:10.842301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:10.887486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1e79-e9d3-4c1e-9f4e-9179ceb3a7b6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:10.917432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 542/1078] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:10.928486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:10.929102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:10.929383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:10.929658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:10.930350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:10.930656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:10.932022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:10.940923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:10.944938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 537/1079] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:10.988442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:11.050458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187f9d8-a260-4855-8087-f41e8d09de69 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:11.087461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:11.088113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:11.088410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:11.088689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:11.100792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:11.105763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:11.106841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:11.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 543/1080] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:11.114927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:11.227403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b3917-9f3b-4d0a-9f00-a23498ed479a None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:11.245310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:11.261895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:11.262521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:11.262803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:11.263077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:11.263791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:11.264098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:11.265519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:11.274658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 538/1081] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:11.280404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:11.375504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:11.378438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a64d-3175-423b-9591-bbe087303489 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:11.412180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:11.412842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:11.413125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:11.413399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] PATH_INFO: `/v3/users/b2c3a753e3d24ca19ca02ec02dd35539` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:11.414552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c3a753e3d24ca19ca02ec02dd35539)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:11.416029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:11.417626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 544/1082] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:11 2022] GET /identity/v3/users/b2c3a753e3d24ca19ca02ec02dd35539 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:11.424218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:11.524405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c4382-10df-42af-a9a1-07148691e41f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:11.557609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:11.558481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:11.558755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:11.559019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:11.560143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:11.561578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428a5af-c34a-4cdd-9df0-bb3fc04aa86b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:11.563252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 539/1083] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:11 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:11.569443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:11.665799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cac5e-3f30-410a-8522-8790a7709246 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:11.699202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:11.699909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:11.700190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:11.700490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f/users/b2c3a753e3d24ca19ca02ec02dd35539/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:11.702429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] RBAC: Authorizing `identity:create_grant(project_id=051f9e3715f74d0d958f3a5eecc90e7f, user_id=b2c3a753e3d24ca19ca02ec02dd35539, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:11.704405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526592a-71ee-440b-b129-5f2e1d94721f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:11.793313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 545/1084] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:15:11 2022] PUT /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f/users/b2c3a753e3d24ca19ca02ec02dd35539/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:15:12.664272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b70-2886-49f8-a463-2397a6b17f28 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:12.664816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b70-2886-49f8-a463-2397a6b17f28 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:12.670142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b70-2886-49f8-a463-2397a6b17f28 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:12.670142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 540/1085] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:12.671384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:12.671757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:12.671957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:12.690403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:12.691680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:12.715046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:12.782540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 546/1086] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:12 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:12.805942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6b70-2886-49f8-a463-2397a6b17f28 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:12.856042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:12.908609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6b70-2886-49f8-a463-2397a6b17f28 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:12.946329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:12.946962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:12.947243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:12.947536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:12.958160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:12.963055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:12.964008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:12.965351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 541/1087] 10.222.0.144 () {56 vars in 1125 bytes} [Tue Jul 26 16:15:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:12.970230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.065820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1c94b-4c12-4faa-b53b-ba1909eccf67 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.098919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.099528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.099810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.100085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.100821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:13.101255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.102634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:13.110893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 547/1088] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:15:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:13.116724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.220201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e7edf-c3d8-41b3-aac1-88aebc9dc15e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.229326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:13.253989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.254619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.254900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.255171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.265541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.266974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:13.281340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:13.282254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:13.283745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:13.283745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:13.283745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:13.283745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:13.283745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 542/1089] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:13.288183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.310759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:13.387074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6116a1-7341-4033-8908-0da90dab8f43 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.420737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.421875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.422159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.422432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.423114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:13.423418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.424799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:13.436455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 548/1090] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:13.442662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.544026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177106f2-4ae1-44cd-8f22-6b45063215f4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.577613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.578237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.578518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.578799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.586145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.587576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ca0ce9fd-4121-4e03-9c4d-1a4a5818c394 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:15:13.599306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:13.600418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:13.601379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 543/1091] 10.222.0.144 () {56 vars in 1098 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:13.606158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.685062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:13.703531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0688df-03c9-431f-aabb-dd728537c8ed None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.748721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.749334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.749787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.750103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.750900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:13.751211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.752659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:13.768412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b1f20-38bc-413b-8179-12a75f379e4f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.768893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b1f20-38bc-413b-8179-12a75f379e4f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.768999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b1f20-38bc-413b-8179-12a75f379e4f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.769318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 549/1092] 10.222.0.144 () {56 vars in 1138 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:13.770295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 544/1093] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:13.774785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.775290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.775619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.775896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.793776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:13.794979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:13.810021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:13.862455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 545/1094] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:13 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:13.873343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceccfbd-7e4e-42ee-9d43-8bb92ee206ea None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.885071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:13.908846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:13.909460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:13.909735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:13.910009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:13.917903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:13.980980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:13.982007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0a306-8bb9-4cc3-aa0a-0f60a7cd7aa7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:13.982345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.023046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.023667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.023944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.024214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.034506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:14.039294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:14.040437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:14.041492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 546/1095] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:14.046272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.058411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.081808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 550/1096] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:13 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:14.087484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.124938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.144119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff4bd-afff-4572-af72-b6037950c2a9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.177552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.178160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.178436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.178706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.179410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:14.179709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.181086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.183549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1c30-2b65-4d63-9b45-31ec9b830abf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.191765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 547/1097] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.197403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.216582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.217211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.217489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.217765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.228084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.229547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:14.243703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:14.244773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:14.245861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:15:14.245861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:14.245861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:15:14.245861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:14.245861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 551/1098] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:14.250964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.268568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.296529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948f05a-0437-4c80-a024-eca5ef092395 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.331342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.331969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.332246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.332548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] PATH_INFO: `/v3/users/b2c3a753e3d24ca19ca02ec02dd35539` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.333714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c3a753e3d24ca19ca02ec02dd35539)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.335122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.336744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 548/1099] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/users/b2c3a753e3d24ca19ca02ec02dd35539 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.338908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.342286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.348013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab4cc8-c436-4818-9c85-bb514d4e1b24 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.381416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.382023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.382307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.382584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.383271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:14.383575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.384994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.397211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 552/1100] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.402842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.450041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e357-66b1-4b24-a49d-6e1ce4898816 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.482886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.483492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.483829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.484073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.499146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.504303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36f3e-724a-40e0-94a2-d466d6c77f7e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.538826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.539452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.539728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.539996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.546509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.547933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.555909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.557374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:15:14.560041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:14.561116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:14.562559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 553/1101] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:15:14.567438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.579594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.658351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 549/1102] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:14.664257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.665154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cb813-0115-496f-a9cd-b4cfc8d3a990 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.701007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.701660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.701897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.702167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.702967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:14.703271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.704711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.716027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 554/1103] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.722012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.724137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.781172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bcb1b-7554-49c6-98c9-51f868926f69 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.788083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.815754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.816398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.816685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.816953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.818086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.819521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.819750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5cd6-7620-4f0f-b054-a2d84bde094f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.821489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 550/1104] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.827429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:14.852814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.853427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.853702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.853973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.854748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] RBAC: Adding query filter params (role.id=95b534da3fb4459096d20252e299be11, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=6268372f3956497aae05aec64d41e848) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:14.855054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.856479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016578-8b88-4460-9316-1e63187362f9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.862558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 555/1105] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/role_assignments?user.id=6268372f3956497aae05aec64d41e848&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=95b534da3fb4459096d20252e299be11 => generated 610 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:14.928442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c16e-7981-468f-991b-e7b5d0499884 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:14.938938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:14.964303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:14.964960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:14.965239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:14.965510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:14.966309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f, user.id=b2c3a753e3d24ca19ca02ec02dd35539) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:14.966621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:14.968064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf8b58-4d46-4ca4-acdb-1ac54c4c0276 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:14.975645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 551/1106] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:14 2022] GET /identity/v3/role_assignments?user.id=b2c3a753e3d24ca19ca02ec02dd35539&scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:16.997166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c05e-2deb-410c-85f2-8e9b6dbd981a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:16.997730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c05e-2deb-410c-85f2-8e9b6dbd981a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:16.997730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c05e-2deb-410c-85f2-8e9b6dbd981a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:16.999036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 556/1107] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:15:17.005066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115262-5b33-447e-8301-d120698f6266 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:17.005499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115262-5b33-447e-8301-d120698f6266 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:17.005601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115262-5b33-447e-8301-d120698f6266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:17.026827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-51115262-5b33-447e-8301-d120698f6266 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:17.028050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-51115262-5b33-447e-8301-d120698f6266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:17.106495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 552/1108] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:17 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:17.131684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40c05e-2deb-410c-85f2-8e9b6dbd981a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:17.234962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40c05e-2deb-410c-85f2-8e9b6dbd981a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:17.267121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:17.267671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:17.267950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:17.268217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:17.269391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:17.270804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638b79a-ce32-4998-b3bc-4110065457d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:17.272365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 557/1109] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:17 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:17.278203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51115262-5b33-447e-8301-d120698f6266 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:17.373114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51115262-5b33-447e-8301-d120698f6266 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:17.374719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:17.406194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:17.406797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:17.407077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:17.407350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:17.408026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:17.409572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820c8f1-a2b4-43d9-9881-2d76da34c6a3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:17.478267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 553/1110] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:15:17 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:19.526284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326dfd3-dda0-4bcb-875d-7eff2a72c3ed None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:19.526800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326dfd3-dda0-4bcb-875d-7eff2a72c3ed None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:19.526898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326dfd3-dda0-4bcb-875d-7eff2a72c3ed None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:19.528175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 558/1111] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:19.534045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:19.534467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:19.534596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:19.553578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:19.554761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:19.619061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 554/1112] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:19 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:19.641482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326dfd3-dda0-4bcb-875d-7eff2a72c3ed None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:19.644311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:19.739769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326dfd3-dda0-4bcb-875d-7eff2a72c3ed None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:19.770889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:19.771456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:19.771731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:19.772000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] PATH_INFO: `/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:19.773413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] RBAC: Authorizing `identity:get_role(role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:19.774770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:19.776074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 559/1113] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:19 2022] GET /identity/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 224 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:19.782369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:19.865620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:19.878466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ab8b0-2f7d-4491-b2a5-df502becf585 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:19.914941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:19.915574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:19.915858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:19.916134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] PATH_INFO: `/v3/users/c5ee41618a9749baab341063ebeea13d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:19.928892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] RBAC: Authorizing `identity:get_user(user_id=c5ee41618a9749baab341063ebeea13d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:19.930316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:19.931936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 555/1114] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:19 2022] GET /identity/v3/users/c5ee41618a9749baab341063ebeea13d => generated 287 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:19.937984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:20.004530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:20.035318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6b6fc-ea8a-42a4-b666-02d2658f369a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:20.067222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:20.067820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:20.068099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:20.068370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:20.132403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:20.133851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:20.197926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:20.228877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 560/1115] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:19 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:20.234651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:20.331801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ec69-630d-449f-ac08-0d3691dd534c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:20.364187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:20.364840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:20.365126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:20.365402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:20.366547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:20.367967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4aba-9b57-40dd-9595-f5833e3655d1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:20.369647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 556/1116] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:20 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:20.376102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:20.471669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24577a5-994a-4e70-8041-52edf0557e2b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:20.506587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:20.507196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:20.507474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:20.507747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:20.508561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f, user.id=c5ee41618a9749baab341063ebeea13d) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:20.508869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:20.510279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e8923-a80c-4f2b-9e5b-e03767451122 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:20.516625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 561/1117] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:20 2022] GET /identity/v3/role_assignments?user.id=c5ee41618a9749baab341063ebeea13d&scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 260 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:22.450352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949336-a0a3-43c7-8e7e-7d5c232a9e81 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:22.450912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949336-a0a3-43c7-8e7e-7d5c232a9e81 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:22.450912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949336-a0a3-43c7-8e7e-7d5c232a9e81 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:22.452271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 557/1118] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:22.457309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:22.457751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:22.457942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:22.476874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:22.478067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:22.506333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:22.547262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 562/1119] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:22 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:22.571006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949336-a0a3-43c7-8e7e-7d5c232a9e81 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:22.586241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:22.668398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949336-a0a3-43c7-8e7e-7d5c232a9e81 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:22.701182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:22.701743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:22.702023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:22.702298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] PATH_INFO: `/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:22.703719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] RBAC: Authorizing `identity:get_role(role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:22.705099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:22.706415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 558/1120] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:22 2022] GET /identity/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 224 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:22.712157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:22.806404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ed423-67c4-4f23-ad34-12306043e81e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:22.838489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:22.839111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:22.839390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:22.839664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] PATH_INFO: `/v3/users/c5ee41618a9749baab341063ebeea13d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:22.840833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ee41618a9749baab341063ebeea13d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:22.842256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:22.844404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 563/1121] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:22 2022] GET /identity/v3/users/c5ee41618a9749baab341063ebeea13d => generated 287 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:22.850324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:22.934441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:22.951352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335517-8c7b-4bb0-b445-c42eead18df4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:22.984940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:22.985560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:22.985838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:22.986109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:22.987240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:22.988678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1370c1-d59f-4a68-bd43-291d0316b43c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:22.990305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 559/1122] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:22 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:22.996268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:23.026954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:23.093541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f466f-4c60-4e18-a278-fb85a4969623 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:23.125323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:23.125969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:23.126258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:23.126535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f/users/c5ee41618a9749baab341063ebeea13d/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:23.128715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] RBAC: Authorizing `identity:create_grant(project_id=051f9e3715f74d0d958f3a5eecc90e7f, user_id=c5ee41618a9749baab341063ebeea13d, role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:23.130263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c7f23-509d-4e62-9d35-66606d007984 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:23.198045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 564/1123] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:15:22 2022] PUT /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f/users/c5ee41618a9749baab341063ebeea13d/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:15:25.151238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6d155-06b8-4b0d-b1be-77e5d09400c4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.151795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6d155-06b8-4b0d-b1be-77e5d09400c4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.151795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6d155-06b8-4b0d-b1be-77e5d09400c4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.153144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 560/1124] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:25.158720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.159116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.159271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.176746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:25.177926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:25.242191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 565/1125] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:15:25.265677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6d155-06b8-4b0d-b1be-77e5d09400c4 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:25.365181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6d155-06b8-4b0d-b1be-77e5d09400c4 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:25.398338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:25.398959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.399234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.399503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] PATH_INFO: `/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.400939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] RBAC: Authorizing `identity:get_role(role_id=d9bc5113c7ec48f48e4625e1c9f7c50a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:25.402347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:25.403655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 561/1126] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:25 2022] GET /identity/v3/roles/d9bc5113c7ec48f48e4625e1c9f7c50a => generated 224 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:25.410093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:25.433084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:25.509952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd3acd-c6e5-460f-b7b1-2ec680aa2f3e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:25.543008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:25.543638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.543917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.544189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] PATH_INFO: `/v3/users/c5ee41618a9749baab341063ebeea13d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.545430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] RBAC: Authorizing `identity:get_user(user_id=c5ee41618a9749baab341063ebeea13d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:25.546887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:25.548487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 566/1127] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:25 2022] GET /identity/v3/users/c5ee41618a9749baab341063ebeea13d => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:25.554268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:25.573339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:25.652103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4592-5920-4843-ac5f-fdea3d7f4b1c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:25.684162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:25.684729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.685010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.685282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.750537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:25.751960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:25.768511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:25.846562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 562/1128] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:25 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:25.852491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:25.940188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:25.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4231e-dd2e-4fdf-855d-40dfaa5b662c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:25.983271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:25.983896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:25.984172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:25.984465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] PATH_INFO: `/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:25.985588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] RBAC: Authorizing `identity:get_project(project_id=051f9e3715f74d0d958f3a5eecc90e7f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:25.987014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402343a-7113-42d2-b86a-a9b89c3220d9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:25.988791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 567/1129] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:25 2022] GET /identity/v3/projects/051f9e3715f74d0d958f3a5eecc90e7f => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:25.994869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:26.090083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3711-d35d-4ec9-9c8f-f1558f709000 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:26.106894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:26.122520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:26.123127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:26.123408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:26.123681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:26.124494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] RBAC: Adding query filter params (role.id=d9bc5113c7ec48f48e4625e1c9f7c50a, scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f, user.id=c5ee41618a9749baab341063ebeea13d) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:26.124860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:26.126275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4274e57-6718-4425-a11c-080feb1ab64a None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:26.131822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 563/1130] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:25 2022] GET /identity/v3/role_assignments?user.id=c5ee41618a9749baab341063ebeea13d&scope.project.id=051f9e3715f74d0d958f3a5eecc90e7f&role.id=d9bc5113c7ec48f48e4625e1c9f7c50a => generated 610 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:28.121427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3afff3-b960-4097-a9fb-e949b8921a61 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:28.121976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3afff3-b960-4097-a9fb-e949b8921a61 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:28.121976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3afff3-b960-4097-a9fb-e949b8921a61 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:28.123342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 568/1131] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:28.128603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:28.128999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:28.129151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:28.147279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:28.148493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:28.212839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 564/1132] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:28 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:28.236237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3afff3-b960-4097-a9fb-e949b8921a61 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:28.333628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3afff3-b960-4097-a9fb-e949b8921a61 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:28.365973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f61549-c187-49dc-8810-27028f0848af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:28.366592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61549-c187-49dc-8810-27028f0848af None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:28.366873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61549-c187-49dc-8810-27028f0848af None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:28.367146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f61549-c187-49dc-8810-27028f0848af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:28.368279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f61549-c187-49dc-8810-27028f0848af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:28.369738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f61549-c187-49dc-8810-27028f0848af None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:28.371278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 569/1133] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:28 2022] GET /identity/v3/domains/default => generated 200 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:28.377338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:28.472276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0e127-9de7-4ab8-a37f-6eb649f44032 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:28.485266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:28.505825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:28.506433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:28.506840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:28.507016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:28.507687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:28.509128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c19261-630d-45ec-80c0-12047af3ceb3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:28.593509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 565/1134] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:15:28 2022] POST /identity/v3/projects => generated 312 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:30.579327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240df9-14df-401c-8856-88e040d8e4ba None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:30.579855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240df9-14df-401c-8856-88e040d8e4ba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:30.579855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240df9-14df-401c-8856-88e040d8e4ba None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:30.581216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 570/1135] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:30.586374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:30.586762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:30.586922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:30.606155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:30.607336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:30.672218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 566/1136] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:30 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:30.696564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4240df9-14df-401c-8856-88e040d8e4ba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:30.723869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:30.795000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4240df9-14df-401c-8856-88e040d8e4ba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:30.827350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:30.828128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:30.828429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:30.828699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:30.829821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:30.831229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94281f7f-ab78-40d9-8006-6e67f5de39c7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:30.832797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 571/1137] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:15:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:30.838517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:30.928508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:30.935234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad95c2e-0170-4b25-90a7-ffc5ba844f66 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:30.967269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:30.967871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:30.968713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:30.968991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:30.969670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:30.971077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b61ed-982b-48d8-980b-9bf93850e9fb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:31.054248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 567/1138] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:15:30 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:33.062230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9565eb-bb6a-4696-b0f6-b99e6f321f08 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.062788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9565eb-bb6a-4696-b0f6-b99e6f321f08 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.062788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9565eb-bb6a-4696-b0f6-b99e6f321f08 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.064098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 572/1139] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:33.069257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.069672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.069796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.087799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:33.088997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:33.152598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 568/1140] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:33 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:33.176140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9565eb-bb6a-4696-b0f6-b99e6f321f08 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:33.267873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:33.275196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9565eb-bb6a-4696-b0f6-b99e6f321f08 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:33.310264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:33.310888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.311168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.311440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.321906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:33.326797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:33.328537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:33.329840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 573/1141] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:33.334895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:33.412594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:33.430418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bbf00-4920-4d72-9511-422556f0d4fe None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:33.464579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:33.465185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.465464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.465734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.466443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:33.466744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:33.468661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:33.476657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 569/1142] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:33 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:33.482332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:33.578317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a841b-c6a8-46e6-a45e-24bd6e92f67a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:33.610933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:33.611548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.611824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.612093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] PATH_INFO: `/v3/users/05ff2162a0714cbea2dc90e294e59051` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.624596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] RBAC: Authorizing `identity:get_user(user_id=05ff2162a0714cbea2dc90e294e59051)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:33.626011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:33.627581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 574/1143] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:33 2022] GET /identity/v3/users/05ff2162a0714cbea2dc90e294e59051 => generated 287 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:33.633475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:33.731519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f26bf-7fad-4aa5-8242-685be718d0ca None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:33.763649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:33.764245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:33.764554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:33.765368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:33.780592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:33.834540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:33.835893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:33.930053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 570/1144] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:33 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:33.936288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:33.951808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:34.033009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e979-bf50-4625-bb5b-465e0a56ee78 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:34.065603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:34.066227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:34.066506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:34.066781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] PATH_INFO: `/v3/projects/85cc11bd95cf43f38f5a64cd915d62df` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:34.067913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] RBAC: Authorizing `identity:get_project(project_id=85cc11bd95cf43f38f5a64cd915d62df)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:34.069360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416a4f-f0ef-40da-8ff7-0cfc85deb901 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:34.070980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 575/1145] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:33 2022] GET /identity/v3/projects/85cc11bd95cf43f38f5a64cd915d62df => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:34.077079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:34.118621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:34.172293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733901ce-a6e6-4c37-a8eb-e1f1ad919a3c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:34.205093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81220-277e-4209-9a60-80378852df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:34.205689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81220-277e-4209-9a60-80378852df35 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:34.205965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81220-277e-4209-9a60-80378852df35 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:34.206240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81220-277e-4209-9a60-80378852df35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:34.207018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e81220-277e-4209-9a60-80378852df35 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=85cc11bd95cf43f38f5a64cd915d62df, user.id=05ff2162a0714cbea2dc90e294e59051) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:34.207327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e81220-277e-4209-9a60-80378852df35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:34.208747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e81220-277e-4209-9a60-80378852df35 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:34.213919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 571/1146] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:34 2022] GET /identity/v3/role_assignments?user.id=05ff2162a0714cbea2dc90e294e59051&scope.project.id=85cc11bd95cf43f38f5a64cd915d62df&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:36.155923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ec5-c2db-478e-8068-06f22a86ad79 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.156496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ec5-c2db-478e-8068-06f22a86ad79 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.156496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ec5-c2db-478e-8068-06f22a86ad79 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.157831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 576/1147] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:36.162999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.163366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.163546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.180985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:36.182163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:36.247217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 572/1148] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:36 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:36.268874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325ec5-c2db-478e-8068-06f22a86ad79 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:36.348099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:36.366700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325ec5-c2db-478e-8068-06f22a86ad79 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:36.399585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:36.400208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.400514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.400788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.411123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:36.415893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:36.416998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:36.417941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 577/1149] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:36.423209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:36.456401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:36.519045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67e69f-2991-498e-aa68-1d2174d26cb0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:36.551492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:36.552093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.552371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.552674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.553377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:36.553677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:36.555015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:36.562966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 573/1150] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:36 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:36.568892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:36.667428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434666c-6b16-479f-b32d-796fda6281f3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:36.699993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:36.700646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.700929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.701199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] PATH_INFO: `/v3/users/05ff2162a0714cbea2dc90e294e59051` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.702835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] RBAC: Authorizing `identity:get_user(user_id=05ff2162a0714cbea2dc90e294e59051)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:36.704247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:36.705874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 578/1151] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:36 2022] GET /identity/v3/users/05ff2162a0714cbea2dc90e294e59051 => generated 287 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:36.711851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:36.803879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:36.807316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adfb4-3be5-42da-87da-9384930cae34 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:36.840920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:36.841533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.841811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.842080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] PATH_INFO: `/v3/projects/85cc11bd95cf43f38f5a64cd915d62df` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.843232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] RBAC: Authorizing `identity:get_project(project_id=85cc11bd95cf43f38f5a64cd915d62df)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:36.844666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e0658-913d-4efb-adb4-9b733fa64635 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:36.846283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 574/1152] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:36 2022] GET /identity/v3/projects/85cc11bd95cf43f38f5a64cd915d62df => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:36.852108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:36.863160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:36.948472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13890462-a72d-46d3-b5fa-a86fa1f28c72 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:36.991699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:36.992365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:36.992679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:36.992958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] PATH_INFO: `/v3/projects/85cc11bd95cf43f38f5a64cd915d62df/users/05ff2162a0714cbea2dc90e294e59051/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:36.994904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] RBAC: Authorizing `identity:create_grant(project_id=85cc11bd95cf43f38f5a64cd915d62df, user_id=05ff2162a0714cbea2dc90e294e59051, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:36.996329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0253ed-1576-4867-a16d-ada7281736dc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:37.100202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 579/1153] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:15:36 2022] PUT /identity/v3/projects/85cc11bd95cf43f38f5a64cd915d62df/users/05ff2162a0714cbea2dc90e294e59051/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:15:39.049791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d000-c8f0-438b-b865-e173abca2c73 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.050325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d000-c8f0-438b-b865-e173abca2c73 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.050325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d000-c8f0-438b-b865-e173abca2c73 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.051662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 575/1154] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:39.056928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.057324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.057830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.075416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:39.076635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:39.135807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:39.141109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 580/1155] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:39 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:39.164481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d000-c8f0-438b-b865-e173abca2c73 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:39.262822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d000-c8f0-438b-b865-e173abca2c73 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:39.294802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:39.295420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.295720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.296547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.306758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:39.311803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:39.312901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:39.313986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 576/1156] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:39.318830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:39.421229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e83301-82ac-4585-8671-c8ddc3c09ae5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:39.458524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:39.459133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.459417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.459696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.460425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:39.460737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:39.462105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:39.470770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 581/1157] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:39 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:39.476613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:39.505069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:39.574429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52cb7e-8f9c-47b4-888d-7f8786f30570 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:39.606800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:39.607418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.607697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.607967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] PATH_INFO: `/v3/users/05ff2162a0714cbea2dc90e294e59051` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.609159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] RBAC: Authorizing `identity:get_user(user_id=05ff2162a0714cbea2dc90e294e59051)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:39.610571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:39.612146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 577/1158] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:39 2022] GET /identity/v3/users/05ff2162a0714cbea2dc90e294e59051 => generated 287 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:39.617855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:39.674284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:39.713939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7f3f-286b-4b32-96ce-47e2cf5725af None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:39.748272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:39.749412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:39.749710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:39.749979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:39.815380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:39.816863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:39.871411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:39.910499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 582/1159] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:39 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:39.916376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:40.005405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:40.011526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d7ee-fc67-4ed6-a0fd-06ef92c0736f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:40.045014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:40.045644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:40.046488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:40.046763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] PATH_INFO: `/v3/projects/85cc11bd95cf43f38f5a64cd915d62df` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:40.047909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] RBAC: Authorizing `identity:get_project(project_id=85cc11bd95cf43f38f5a64cd915d62df)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:40.049364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159ab9-edfc-4361-8977-b58bd9dc8b57 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:40.051014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 578/1160] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:39 2022] GET /identity/v3/projects/85cc11bd95cf43f38f5a64cd915d62df => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:40.057091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:40.150844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bcd8c-035d-4c7c-956a-a046cfdea95f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:40.183291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:40.183887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:40.184166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:40.184466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:40.185245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=85cc11bd95cf43f38f5a64cd915d62df, user.id=05ff2162a0714cbea2dc90e294e59051) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:40.185554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:40.186991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e582-1139-4454-b00c-fc85d3f26639 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:40.192521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 583/1161] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:40 2022] GET /identity/v3/role_assignments?user.id=05ff2162a0714cbea2dc90e294e59051&scope.project.id=85cc11bd95cf43f38f5a64cd915d62df&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:42.190841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c660-c6ce-4298-b35a-3208a7135be0 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:42.191374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c660-c6ce-4298-b35a-3208a7135be0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:42.191374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66c660-c6ce-4298-b35a-3208a7135be0 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:42.192758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 579/1162] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:42.197988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:42.198419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:42.198545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:42.216656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:42.217841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:42.234265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:42.282065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 584/1163] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:42 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:42.303727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c660-c6ce-4298-b35a-3208a7135be0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:42.403272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66c660-c6ce-4298-b35a-3208a7135be0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:42.435615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:42.436843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:42.437128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:42.437397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:42.465232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:42.503200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:42.504690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:42.599689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 580/1164] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:42 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:42.605633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:42.700036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6446e-a197-4d5b-9c8e-e955b429bc59 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:42.732695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:42.733812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:42.734086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:42.734351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:42.740852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:42.742283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f9015a31-2afb-48f3-9f40-7527b298810a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:42.753423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:42.754619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:42.755664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 585/1165] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:15:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:42.760960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:42.820969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:42.858969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39add8-b20b-4346-ad93-fb4d1b173ba6 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:42.890856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:42.891452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:42.891728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:42.891997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:42.892704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:42.893006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:42.894477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b53b84-7803-41ff-a98a-78c78d82aafb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:42.901293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 581/1166] 10.222.0.144 () {56 vars in 1120 bytes} [Tue Jul 26 16:15:42 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:44.840049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9fb4-404b-42f9-9dc5-7dad88a4c2c5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:44.840580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9fb4-404b-42f9-9dc5-7dad88a4c2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:44.840580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9fb4-404b-42f9-9dc5-7dad88a4c2c5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:44.841882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 586/1167] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:44.846871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984fb0-e54b-4705-b3d6-5e2aafd0f25c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:44.847302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984fb0-e54b-4705-b3d6-5e2aafd0f25c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:44.847421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47984fb0-e54b-4705-b3d6-5e2aafd0f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:44.865497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-47984fb0-e54b-4705-b3d6-5e2aafd0f25c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:44.866667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-47984fb0-e54b-4705-b3d6-5e2aafd0f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:44.930801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 582/1168] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:44 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:44.953955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9fb4-404b-42f9-9dc5-7dad88a4c2c5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:44.970018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:45.052465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9fb4-404b-42f9-9dc5-7dad88a4c2c5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:45.084534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:45.085129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:45.085405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:45.085673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:45.086360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:45.087694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b46b37-f2b7-4dd1-9f12-4fb35e41a69b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:45.140624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 587/1169] 10.222.0.144 () {60 vars in 1143 bytes} [Tue Jul 26 16:15:44 2022] POST /identity/v3/domains => generated 268 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:47.139369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e0543-f21e-4d4a-93a8-2242c425e972 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:47.139907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e0543-f21e-4d4a-93a8-2242c425e972 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:47.139907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e0543-f21e-4d4a-93a8-2242c425e972 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:47.141312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 583/1170] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:47.146570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:47.146962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:47.147125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:47.165226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:47.166422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:47.232289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 588/1171] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:47 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:47.255338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e0543-f21e-4d4a-93a8-2242c425e972 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:47.311369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:47.353873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e0543-f21e-4d4a-93a8-2242c425e972 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:47.387044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:47.387667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:47.387946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:47.388219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] PATH_INFO: `/v3/domains/f38ebf7056a9420dbf422eda6a0cb6e1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:47.398831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=f38ebf7056a9420dbf422eda6a0cb6e1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:47.400210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63e2c2-6dad-42f2-aa78-0ad1e2ce69dd None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:47.402788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 584/1172] 10.222.0.144 () {56 vars in 1188 bytes} [Tue Jul 26 16:15:47 2022] GET /identity/v3/domains/f38ebf7056a9420dbf422eda6a0cb6e1 => generated 268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:47.408685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:47.411728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:47.512163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd5563-1c2c-463a-861a-dd7163061c2c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:47.545153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:47.545765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:47.546041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:47.546309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:47.546989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:47.548435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a62c-cdaf-48d8-a58b-6364a0adfef0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:47.631653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 589/1173] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:15:47 2022] POST /identity/v3/projects => generated 362 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:49.644608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ed1-9803-4955-97f9-4d0b58bf6945 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:49.645191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ed1-9803-4955-97f9-4d0b58bf6945 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:49.645191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ed1-9803-4955-97f9-4d0b58bf6945 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:49.646537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 585/1174] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:49.651549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:49.651944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:49.652096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:49.670886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:49.672057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:49.719852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:49.735953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 590/1175] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:49 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:49.759861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d7ed1-9803-4955-97f9-4d0b58bf6945 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:49.858704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d7ed1-9803-4955-97f9-4d0b58bf6945 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:49.873829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:49.890907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:49.891529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:49.891809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:49.892081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] PATH_INFO: `/v3/domains/f38ebf7056a9420dbf422eda6a0cb6e1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:49.893252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f38ebf7056a9420dbf422eda6a0cb6e1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:49.894661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2fb1b-516b-46c5-8740-01e52d036518 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:49.896229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 586/1176] 10.222.0.144 () {56 vars in 1188 bytes} [Tue Jul 26 16:15:49 2022] GET /identity/v3/domains/f38ebf7056a9420dbf422eda6a0cb6e1 => generated 268 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:49.902227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:49.996505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4a007-8c75-4274-ba3f-9090f4d0f716 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:50.029618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:50.030665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:50.030945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:50.031214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:50.031878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:50.033320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131335dc-8129-4517-bdce-eaf747cd7e18 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:50.133114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 591/1177] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:15:49 2022] POST /identity/v3/users => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:15:52.141218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaf9cb-95e2-4443-aaf3-80321f23a97c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.141767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaf9cb-95e2-4443-aaf3-80321f23a97c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.141767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaf9cb-95e2-4443-aaf3-80321f23a97c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.143127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 587/1178] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:52.148377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.148787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.148951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.166964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:52.168143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:52.202205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:52.232878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 592/1179] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:52 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:52.255573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaf9cb-95e2-4443-aaf3-80321f23a97c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:52.354367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaf9cb-95e2-4443-aaf3-80321f23a97c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:52.385665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:52.386279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.386559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.386830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.397618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:52.402421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:52.403596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:52.405190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 588/1180] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:52.410071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:52.515648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce85dd-779f-471e-843f-4368e553fd21 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:52.548012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:52.548644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.548927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.549201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.550049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:52.550348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:52.551719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:52.559768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 593/1181] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:52.566063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:52.609785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:52.661963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ff1da-9cfe-4ca5-8a73-36dac159d232 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:52.693971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:52.695656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.695656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.695656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] PATH_INFO: `/v3/users/915da04331b2468295f45f47a7e2e79a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.708235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] RBAC: Authorizing `identity:get_user(user_id=915da04331b2468295f45f47a7e2e79a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:52.709710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:52.711294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 589/1182] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:52 2022] GET /identity/v3/users/915da04331b2468295f45f47a7e2e79a => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:52.717056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:52.750725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:52.813434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4cbf9-0564-49c5-92e5-507b5f1a306e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:52.845472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:52.846083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:52.846405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:52.846707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:52.914598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:52.915961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:52.947660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:53.010236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 594/1183] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:52 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:53.016205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:53.111512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4ec9-4aad-4ddd-ac3a-29b4a822f211 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:53.113019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:53.145725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:53.146328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:53.146610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:53.146878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] PATH_INFO: `/v3/projects/2158447f004d40118d677650e6bb48a1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:53.148003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] RBAC: Authorizing `identity:get_project(project_id=2158447f004d40118d677650e6bb48a1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:53.149449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31132c9b-12c0-4aa9-99a0-215cb663b849 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:53.151073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 590/1184] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:53 2022] GET /identity/v3/projects/2158447f004d40118d677650e6bb48a1 => generated 362 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:53.157020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:53.251841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e82b40-a1f8-4ac3-a79f-beaf04090ed3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:53.284221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:53.284847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:53.285128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:53.285400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:53.286177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=2158447f004d40118d677650e6bb48a1, user.id=915da04331b2468295f45f47a7e2e79a) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:53.286490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:53.287901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1509f-c10e-4305-a7e1-82d8c17f8d0b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:53.289649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:53.293280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 595/1185] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:53 2022] GET /identity/v3/role_assignments?user.id=915da04331b2468295f45f47a7e2e79a&scope.project.id=2158447f004d40118d677650e6bb48a1&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:55.270406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34939b-a65b-4881-b176-7ec5bcfb8007 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.270955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34939b-a65b-4881-b176-7ec5bcfb8007 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.270955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34939b-a65b-4881-b176-7ec5bcfb8007 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.272296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 591/1186] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:55.277319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.277732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.277866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.296749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:55.297928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:55.361505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 596/1187] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:55 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:55.383269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34939b-a65b-4881-b176-7ec5bcfb8007 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:55.482164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34939b-a65b-4881-b176-7ec5bcfb8007 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:55.515972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:55.516636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.516918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.517189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.526927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:55.531762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:55.532850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:15:55.533882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 592/1188] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:55.538889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:55.636865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fef3d-5be0-4a0a-93c9-df6585a23130 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:55.662691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:55.670258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:55.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:55.683021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 597/1189] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:55 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:55.688050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:55.718682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:55.785149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790e9e5-0972-4a63-bc51-667129f62034 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:55.817858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:55.818483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.818764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.819037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] PATH_INFO: `/v3/users/915da04331b2468295f45f47a7e2e79a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.820185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] RBAC: Authorizing `identity:get_user(user_id=915da04331b2468295f45f47a7e2e79a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:55.821630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:55.823217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 593/1190] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:55 2022] GET /identity/v3/users/915da04331b2468295f45f47a7e2e79a => generated 312 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:15:55.829557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:55.927237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a123-8342-4e47-b73e-01b1db6894cb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:55.959157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:55.959773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:55.960052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:55.960322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] PATH_INFO: `/v3/projects/2158447f004d40118d677650e6bb48a1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:55.961477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] RBAC: Authorizing `identity:get_project(project_id=2158447f004d40118d677650e6bb48a1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:55.962889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a230-f435-4620-ab7a-2030d16da979 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:55.964523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 598/1191] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:55 2022] GET /identity/v3/projects/2158447f004d40118d677650e6bb48a1 => generated 362 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:55.971068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:56.062218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:56.065913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eef41-c0e0-4771-9c6c-cab503c37e5e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:56.100078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:56.100753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:56.101034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:56.101305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] PATH_INFO: `/v3/projects/2158447f004d40118d677650e6bb48a1/users/915da04331b2468295f45f47a7e2e79a/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:56.103211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2158447f004d40118d677650e6bb48a1, user_id=915da04331b2468295f45f47a7e2e79a, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:56.104716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab2232-7747-4a10-9ea6-780f7fba02b1 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:56.155943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 594/1192] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:15:55 2022] PUT /identity/v3/projects/2158447f004d40118d677650e6bb48a1/users/915da04331b2468295f45f47a7e2e79a/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:15:58.110722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1455cf-a138-4c10-8444-b901acd68a6d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.111286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1455cf-a138-4c10-8444-b901acd68a6d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.111286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1455cf-a138-4c10-8444-b901acd68a6d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.112635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 599/1193] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:15:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:15:58.117876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedbe46-236b-4245-83db-842c64a02921 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.118297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedbe46-236b-4245-83db-842c64a02921 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.120268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedbe46-236b-4245-83db-842c64a02921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.155972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-daedbe46-236b-4245-83db-842c64a02921 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:15:58.158220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-daedbe46-236b-4245-83db-842c64a02921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:15:58.240154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 595/1194] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:15:58 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:15:58.263761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1455cf-a138-4c10-8444-b901acd68a6d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:58.326260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:58.367093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1455cf-a138-4c10-8444-b901acd68a6d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:58.398923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:58.399539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.399820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.400094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.410267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:15:58.415113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:15:58.415995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:15:58.417127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 600/1195] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:15:58.422862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedbe46-236b-4245-83db-842c64a02921 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:58.509151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:58.519692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedbe46-236b-4245-83db-842c64a02921 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:58.553994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:58.554597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.554877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.555149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.555854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:58.556158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:58.557804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:58.565584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 596/1196] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:15:58 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:58.572401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:58.672742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bacf2-a413-4a7a-ad18-d614798b8c59 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:58.689116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:58.705026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:58.705649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.705929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.706202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] PATH_INFO: `/v3/users/915da04331b2468295f45f47a7e2e79a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.707354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] RBAC: Authorizing `identity:get_user(user_id=915da04331b2468295f45f47a7e2e79a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:58.708805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:58.710383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 601/1197] 10.222.0.144 () {56 vars in 1182 bytes} [Tue Jul 26 16:15:58 2022] GET /identity/v3/users/915da04331b2468295f45f47a7e2e79a => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:58.716417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:58.815357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297eb00a-daec-4d7f-a0c9-5f038cb27542 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:58.848257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:58.848898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:58.849176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:58.849446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:58.868075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:58.915330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:58.916778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:59.013120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 597/1198] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:15:58 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:15:59.019699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:59.116763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90033c2-5c7b-4bb4-bbfc-e441d5f3c960 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:59.149082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:59.149705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:59.149988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:59.150266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] PATH_INFO: `/v3/projects/2158447f004d40118d677650e6bb48a1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:59.151404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] RBAC: Authorizing `identity:get_project(project_id=2158447f004d40118d677650e6bb48a1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:59.152864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0045c-1666-405f-8146-7b045b1daad1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:59.154549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 602/1199] 10.222.0.144 () {56 vars in 1191 bytes} [Tue Jul 26 16:15:59 2022] GET /identity/v3/projects/2158447f004d40118d677650e6bb48a1 => generated 362 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:15:59.160588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:15:59.208882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:15:59.256283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24290799-17a4-4d71-96ae-6681c507a027 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:15:59.289473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:15:59.290015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:15:59.290294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:15:59.290566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:15:59.291345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=2158447f004d40118d677650e6bb48a1, user.id=915da04331b2468295f45f47a7e2e79a) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:15:59.291652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:15:59.293094 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d8045-98e3-4955-aa1d-6f6b87bfe4f6 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:15:59.299459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 598/1200] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:15:59 2022] GET /identity/v3/role_assignments?user.id=915da04331b2468295f45f47a7e2e79a&scope.project.id=2158447f004d40118d677650e6bb48a1&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:16:09.144984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=73 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:16:09.296751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=43 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:10.664758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5e3a-ba84-4df8-aac2-78aa03992629 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:10.665338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5e3a-ba84-4df8-aac2-78aa03992629 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:10.665338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5e3a-ba84-4df8-aac2-78aa03992629 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:10.666668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 603/1201] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:10.671856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:10.672259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:10.672448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:10.691126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:10.693760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:10.758377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 599/1202] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:10.783195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b5e3a-ba84-4df8-aac2-78aa03992629 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:10.887777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b5e3a-ba84-4df8-aac2-78aa03992629 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:10.921136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:10.921764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:10.922048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:10.922328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:10.932843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:10.934309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:10.948913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:10.950016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:10.951357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:10.951357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:10.951357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:10.951357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:10.951357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 604/1203] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:10.956732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:11.051795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2ed1-761b-4a04-9ee6-9353c1b21c44 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:11.079919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:11.085386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:11.085999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:11.086279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:11.086550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:11.087256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:11.087546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:11.088937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864faea5-4568-40ee-aae6-40be32326466 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:11.099610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 600/1204] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:11.105570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:11.158233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:11.204207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe0547-5c2a-47f6-86b1-575d861171f9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:11.237279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:11.237883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:11.238187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:11.238454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:11.239126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:11.240575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21629f-7143-4ff4-a551-79cc16ed5a56 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:11.329821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 605/1205] 10.222.0.144 () {60 vars in 1137 bytes} [Tue Jul 26 16:17:11 2022] POST /identity/v3/users => generated 252 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:17:13.376270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4df9-0028-4afc-b2a5-b6165f1d9911 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:13.376963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4df9-0028-4afc-b2a5-b6165f1d9911 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:13.376963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4df9-0028-4afc-b2a5-b6165f1d9911 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:13.378227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 601/1206] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:13.383404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:13.383794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:13.383958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:13.402382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:13.403606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:13.468336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 606/1207] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:13 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:13.491313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e4df9-0028-4afc-b2a5-b6165f1d9911 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:13.593781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e4df9-0028-4afc-b2a5-b6165f1d9911 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:13.627470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:13.628057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:13.628334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:13.628632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:13.639480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:13.645018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:13.649273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:13.650308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 602/1208] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:13.652458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:13.692683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:13.750555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186de3f-8b01-4735-8cb4-03f1a9c22857 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:13.783513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:13.784113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:13.784409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:13.784681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:13.785378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:13.785677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:13.787027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:13.794538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 607/1209] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:17:13 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:13.804281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:13.855897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:13.907126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4c57-f53f-4c0e-bf6d-dd319606ab6c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:13.940759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:13.941390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:13.941665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:13.941933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:13.952134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:13.956215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:13.968996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:13.970029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:13.970933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:13.970933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:13.970933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:13.970933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:13.970933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 603/1210] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:13.976275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.073332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:14.076186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ff57-6e43-4dbe-8f46-8d8cd5149876 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:14.107895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:14.108523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:14.108806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:14.109080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:14.109758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:14.110059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:14.111413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:14.123686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 608/1211] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:14.130029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.228823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5367de-02d6-4bf8-846a-e729c80098de None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:14.230543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:14.261548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:14.262177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:14.262455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:14.262722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:14.270594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:14.272042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:14.283986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:14.284984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:14.285920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 604/1212] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:14.291237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.386967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b623cd8-17ec-4553-b54a-c631c62e2f79 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:14.420113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:14.420723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:14.421001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:14.421272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:14.422051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:14.422353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:14.423756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:14.441490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 609/1213] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:14.447220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.546707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948af58-9718-4464-8234-b689d1a331c9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:14.584594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:14.585202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:14.585549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:14.585759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:14.613912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:14.651526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:14.653186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:14.751320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 605/1214] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:17:14 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:17:14.757605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.776787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:14.859387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736a24e-4c0b-4876-8307-882f919a6d51 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:14.892997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:14.893754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:14.894059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:14.894328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:14.903944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:14.905579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:14.919686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:14.922166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:14.923231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:14.923231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:14.923231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:14.923231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:14.923231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 610/1215] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:14.927093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:14.990679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:15.027567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044014c-5392-4861-9714-2227b8eb6538 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:15.060274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:15.060941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:15.061219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:15.061494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:15.062180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:15.062484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:15.063855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:15.075316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 606/1216] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:15.081263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:15.148287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:15.178404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0b7d-8f8b-46af-96dc-98bb08362d70 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:15.211372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:15.212481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:15.212986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:15.213272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:15.220174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:15.222234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:15.233365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:15.234429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:15.235599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 611/1217] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:15.241071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:15.338067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999937f8-33f2-4f79-8e98-508354403bbc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:15.360470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:15.372297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:15.372936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:15.373217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:15.373490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:15.374240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:15.374546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:15.375956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c342f-7e21-49e0-bf72-db47574feaf9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:15.387388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 607/1218] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:17:15.393631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:15.492774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57207193-79a9-4238-ba2e-dc3e02db2715 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:15.521124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:15.526944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:15.527557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:15.527839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:15.528113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:15.528934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=8c7f8c220a3844469072a97f7d2cb6e2) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:15.529247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:15.530665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd31413-b445-4e6e-8e05-c05834f9ee6b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:15.535977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 612/1219] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:17:15 2022] GET /identity/v3/role_assignments?user.id=8c7f8c220a3844469072a97f7d2cb6e2&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 260 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:17.521019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd04e-c7c2-46e8-8562-ab7cfb866d23 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:17.521615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd04e-c7c2-46e8-8562-ab7cfb866d23 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:17.521615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd04e-c7c2-46e8-8562-ab7cfb866d23 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:17.522883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 608/1220] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:17.527985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:17.529648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:17.529648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:17.546967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:17.548146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:17.612588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 613/1221] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:17 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:17.634785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bd04e-c7c2-46e8-8562-ab7cfb866d23 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:17.735060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bd04e-c7c2-46e8-8562-ab7cfb866d23 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:17.768533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:17.769099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:17.769376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:17.769644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:17.779233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:17.784095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:17.785623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:17.786444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 609/1222] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:17.791251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:17.891152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b0b32-25e0-4ee3-b05d-b6d0dfe2690b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:17.913427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:17.923699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:17.924862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:17.925149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:17.925421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:17.926123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:17.926426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:17.927778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:17.935602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 614/1223] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:17:17 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:17.942121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:17.985930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:18.041322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab147f-8865-4ea6-895e-ff0ab9b245eb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.074983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.075625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.075907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.076184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.086647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.088082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:18.103050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:18.103975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:18.104880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:18.104880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:18.104880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:18.104880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:18.104880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 610/1224] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:18.110120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.211014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513dfb-ca8f-4fc1-8f63-e1a27416637c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.244688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.245289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.245578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.245853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.246631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:18.246830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.248258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.259765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 615/1225] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:17:18.265704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.358987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:18.365842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875392-8433-4669-9b9d-07fee982490d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.398121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.398771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.399050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.399320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.407040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.409237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:18.420862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:18.423304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:18.425046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 611/1226] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:18.428488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.475221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:18.527554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbf24-93fd-4e0a-898c-a2f6c6baf965 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.562428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.563116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.563366 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.563641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.564456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:18.564769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.566183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.583414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 616/1227] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:18.589152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.700480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302021b8-6a1e-4001-ac05-e75d41cf5d79 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.734633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.735256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.735536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.735805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.746020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.747438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:18.761767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:18.765539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:18.767229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:18.767229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:18.767229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:18.767229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:18.767229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 612/1228] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:18.769272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.881479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2b378-7a81-4ded-8383-5891121929a0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:18.883852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:18.918613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:18.919227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:18.919538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:18.919925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:18.920607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:18.925254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:18.925254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:18.934573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 617/1229] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:18.941255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:18.951479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:19.045425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208fe42-cbd3-4d2e-97fa-15eac406babc None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:19.082282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:19.082918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:19.083206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:19.083478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:19.090080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:19.091529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:19.104652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:19.105710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:19.106760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 613/1230] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:19.112552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:19.217596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103dbdd8-8ebf-4781-bdc0-1432a71a4252 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:19.258286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:19.258917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:19.259207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:19.259482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:19.261420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:19.261735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:19.263252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d0c94-8488-4b69-8a05-079e036977de None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:19.279674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 618/1231] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:19.286248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:19.354121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:19.389075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c53b-5b5b-40ab-a38b-625933c99dd9 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:19.422971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:19.423654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:19.423944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:19.424218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/8c7f8c220a3844469072a97f7d2cb6e2/roles/d14448d4ebec4c5393661ee2b3a19513` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:19.439373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=8c7f8c220a3844469072a97f7d2cb6e2, role_id=d14448d4ebec4c5393661ee2b3a19513)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:19.440910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb5ba6-19aa-4e2f-a504-6eca89325f31 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:19.500837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 614/1232] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:17:19 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/8c7f8c220a3844469072a97f7d2cb6e2/roles/d14448d4ebec4c5393661ee2b3a19513 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:17:21.512287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32eb5b-d8ed-4c88-9ca9-aeef52ece20f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:21.512832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32eb5b-d8ed-4c88-9ca9-aeef52ece20f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:21.512929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32eb5b-d8ed-4c88-9ca9-aeef52ece20f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:21.514194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 619/1233] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:21.519946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:21.520432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:21.520533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:21.538930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:21.540117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:21.605402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 615/1234] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:21 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:21.629038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32eb5b-d8ed-4c88-9ca9-aeef52ece20f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:21.653241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:21.727373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32eb5b-d8ed-4c88-9ca9-aeef52ece20f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:21.760799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:21.761339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:21.761613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:21.761879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:21.771680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:21.776569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:21.781346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:21.782444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 620/1235] 10.222.0.144 () {56 vars in 1107 bytes} [Tue Jul 26 16:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:21.783739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:21.830013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:21.881863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30fd7c-c6f5-4526-bcb6-579b6d698abf None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:21.915358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:21.916350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:21.916930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:21.917414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:21.918712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] RBAC: Adding query filter params (name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:21.919283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:21.921902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:21.937419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 616/1236] 10.222.0.144 () {56 vars in 1108 bytes} [Tue Jul 26 16:17:21 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:21.943803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.029063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:22.044243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d267876-f858-4928-9258-3e8cf2a86600 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:22.078297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:22.078919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:22.079200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:22.079469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:22.090058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:22.093519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:22.105848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:22.106882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:22.107796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:22.107796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:22.107796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:22.107796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:22.107796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 621/1237] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:22.112830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.215561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373173b-fa46-4a37-b7eb-44dd8a59f37d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:22.224526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:22.248233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:22.248957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:22.249233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:22.249500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:22.250177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:22.250514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:22.252479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:22.263837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 617/1238] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:22.270139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.367401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c745f4-f311-4193-bdc8-9ad477789ae5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:22.399812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:22.400572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:22.400852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:22.401122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:22.403845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:22.408822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:22.410433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:22.421525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:22.422644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:22.424133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 622/1239] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:22.429140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.529157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4170606-b5ee-49ae-87a4-3370d9077770 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:22.562397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:22.563008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:22.563290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:22.563563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:22.564932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:22.571747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:22.571747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:22.583798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 618/1240] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:22.589767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.699686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a61024-08a4-41db-afe6-3fcdd63ea852 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:22.732594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:22.733201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:22.733481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:22.733752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:22.792669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:22.799508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:22.800985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:22.894310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 623/1241] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:17:22 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:17:22.900147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:22.927834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:22.998786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576a398-9983-4c8f-b22d-666a2b12b6d0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:23.032081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:23.032737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:23.033540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:23.033810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:23.043575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:23.045033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:23.060564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:23.061442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:23.062312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:23.062312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:23.062312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:23.062312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:23.062312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 619/1242] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:23.067349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:23.160319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:23.166034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555e4a-77a9-48c6-9ba5-6e628b7208ae None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:23.198079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:23.198687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:23.198969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:23.199244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:23.199922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:23.200226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:23.201726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:23.213336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 624/1243] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:23.218907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:23.297368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:23.317709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1726-a975-4171-baf6-5c85afb67671 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:23.350054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:23.350675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:23.350953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:23.351222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:23.357962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:23.359426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:23.371818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:23.372999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:23.374348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 620/1244] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:23.378784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:23.477285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d6c4-fee6-49a3-a97f-16a78425d2a0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:23.510338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:23.510957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:23.511235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:23.511506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:23.512246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:23.512584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:23.514160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829fb37-fe94-4728-b119-65bc84fc65c0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:23.526383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 625/1245] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:23.532544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:23.632402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d3e7-e21a-4059-b5ed-2232581e12eb None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:23.665846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:23.666460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:23.666744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:23.667021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:23.667811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] RBAC: Adding query filter params (role.id=d14448d4ebec4c5393661ee2b3a19513, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=8c7f8c220a3844469072a97f7d2cb6e2) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:23.668127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:23.669583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96827e-8671-49f6-92b8-d0aa7e718c5b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:23.675267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 621/1246] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:17:23 2022] GET /identity/v3/role_assignments?user.id=8c7f8c220a3844469072a97f7d2cb6e2&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=d14448d4ebec4c5393661ee2b3a19513 => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:25.723151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6b8a-f775-4468-b292-8cb775af64e2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:25.723675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6b8a-f775-4468-b292-8cb775af64e2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:25.723785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6b8a-f775-4468-b292-8cb775af64e2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:25.725125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 626/1247] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:25.730132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:25.730522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:25.730692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:25.737574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:25.749677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:25.750840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:25.817443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 622/1248] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:25.840687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a6b8a-f775-4468-b292-8cb775af64e2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:25.859020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:25.940209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a6b8a-f775-4468-b292-8cb775af64e2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:25.974336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:25.974973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:25.975253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:25.975532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:25.990276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:25.996570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:25.997806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:25.998668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 627/1249] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:17:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:26.004205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.102968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eed316-20dd-473d-b4a6-f60a085e5e9f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.131105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:26.135881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.136521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.136806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.137079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.137788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:26.138090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:26.139497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:26.147818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 623/1250] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:26.153598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.230280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:26.251970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8dd7-86cd-4c3d-9604-8cb307a9848e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.284489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.285065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.285344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.285614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.295801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:26.297262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:26.312254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:26.313420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:26.315137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:26.315137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:26.315137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:26.315137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:26.316535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 628/1251] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:26.319963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.425050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c4eb-f8ad-4057-96dc-4eb346e0b588 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.457480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.458096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.458383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.458659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.459352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:26.459659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:26.461070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:26.474192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 624/1252] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:26.479917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.578477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35e4ef-79de-471d-bbde-70a1c03ac78e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.603911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:26.612230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.612866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.613144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.613411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.620718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:26.622126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:26.633519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:26.634514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:26.635473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 629/1253] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:26.640522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.700219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:26.743554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd868bf-bd86-4d0f-b663-56bfcbf8e3e4 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.776508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.777119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.777401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.777676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.778464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:26.778770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:26.780199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:26.797409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 625/1254] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:26.803127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:26.900020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee06ea5-0804-4d8b-9765-354ae344f860 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:26.932640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:26.933245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:26.933529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:26.933808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:26.987893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:26.999945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.001484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.096011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 630/1255] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:17:26 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:17:27.101916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:27.203386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c393-848f-40ab-86ce-637873aaaf91 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:27.216408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:27.237613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:27.238242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:27.238520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:27.238790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:27.249014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.251325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:27.264802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:27.265852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:27.266939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:27.266939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:27.266939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:27.266939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:27.268283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 626/1256] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:27.271890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:27.357814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:27.369963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce27a14-1b8a-43ac-925b-3f06f89096af None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:27.405843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:27.406453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:27.406731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:27.406999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:27.407702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:27.408003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.409405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.420536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 631/1257] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:27.426487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:27.528157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65253b3-7de0-429f-849a-66cd32e323e0 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:27.561986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:27.562627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:27.562906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:27.563181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:27.570407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.571839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:27.582918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:27.584901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:27.586070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 627/1258] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:27.590237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:27.688636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00832579-cb12-4e2d-a28e-c70128e322e9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:27.724460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:27.725068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:27.725346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:27.725617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:27.726351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:27.726655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.729783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe418-3fc9-49b2-878a-965a6ba69be8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.739781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 632/1259] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:27.746030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:27.772500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:27.846431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac051e-54e9-4b39-bee8-6ebb8574824f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:27.879155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:27.879791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:27.880077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:27.880351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:27.881168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=8c7f8c220a3844469072a97f7d2cb6e2) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:27.881481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:27.882943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0c88-1250-496e-a39f-b632f621acfb None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:27.888371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 628/1260] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:17:27 2022] GET /identity/v3/role_assignments?user.id=8c7f8c220a3844469072a97f7d2cb6e2&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 260 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:29.868443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351755b-ede0-49b2-a497-cb5aba91877a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:29.868966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351755b-ede0-49b2-a497-cb5aba91877a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:29.869060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351755b-ede0-49b2-a497-cb5aba91877a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:29.870569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 633/1261] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:17:29.875867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:29.876304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:29.876515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:29.895387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:29.896597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:29.961413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 629/1262] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:29 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:29.983754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351755b-ede0-49b2-a497-cb5aba91877a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:29.994593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.088368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351755b-ede0-49b2-a497-cb5aba91877a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.122042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.122651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.122927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.123198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.133758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:30.138568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:30.139687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:30.146795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 634/1263] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:30.147626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:30.164771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.244878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0ea3f-7fc9-472f-9690-f424bfa549ee None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.277155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.277758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.278038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.278311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.279013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:30.279314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:30.280766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:30.289396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 630/1264] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:30.294935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:30.359862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.391617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e04-6b10-46ab-9702-facdac5b18bb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.424555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.425188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.425473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.425745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.435807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:30.437332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:30.455745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:30.461980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:30.463146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:30.463146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:30.463146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:30.463146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:30.463146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 635/1265] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:30.463499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:30.544305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.564851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380ee3f-686c-458c-ac53-17f86edc703b None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.598861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.599476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.599762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.600037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.600761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:30.601071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:30.602432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:30.614324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 631/1266] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:30.620099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:30.720088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb619c1-b572-422c-a9d7-37de49883309 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.740056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.753966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.754702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.754980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.755250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.763792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:30.765270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:30.776721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:30.778485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:30.779703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 636/1267] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:30.784311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:30.880713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bfd88-034b-4296-a7d3-c8c931f6e35c None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:30.914210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:30.914817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:30.915096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:30.915371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:30.916160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:30.916511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:30.918070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:30.926672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:30.935050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 632/1268] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:30.941341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:31.039881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2517-0499-4c33-a363-bc718c025331 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:31.072895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:31.073528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:31.073804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:31.074075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:31.084884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:31.086422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:31.100319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:31.101367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:31.102271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:31.102271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:31.102271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:31.102271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:31.102271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 637/1269] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:31.107247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:31.204251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43476ef-0f80-4b13-a81d-d25a99b21a76 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:31.240060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:31.240722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:31.241004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:31.241275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:31.241962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:31.242265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:31.243643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:31.255175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 633/1270] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:31.261151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:31.292230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:31.359030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51cc46-724d-4113-8fcf-605b86d689ec None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:31.392608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:31.393234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:31.393514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:31.393785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:31.400623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:31.402051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:31.413586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:31.414695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:31.415568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 638/1271] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:17:31.420758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:31.480997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:31.521612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8174f-5232-4eaf-ba71-3be5e2ba9daa None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:31.554119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:31.554656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:31.554934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:31.555203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:31.556204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:31.556556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:31.557970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcd6cc-1765-4503-b5e1-670a931c1933 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:31.570360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 634/1272] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:31.577973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:31.667507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:31.676957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50bb72-cc9f-4372-9d20-d2edd4a5dfe5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:31.709273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:31.710018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:31.710274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:31.710546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] PATH_INFO: `/v3/projects/dc753d253044415fbf36fa712bea0602/users/8c7f8c220a3844469072a97f7d2cb6e2/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:31.712490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc753d253044415fbf36fa712bea0602, user_id=8c7f8c220a3844469072a97f7d2cb6e2, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:31.713922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59198d02-2a31-42a6-a592-6c77679bc2f8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:31.809559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 639/1273] 10.222.0.144 () {58 vars in 1444 bytes} [Tue Jul 26 16:17:31 2022] PUT /identity/v3/projects/dc753d253044415fbf36fa712bea0602/users/8c7f8c220a3844469072a97f7d2cb6e2/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:17:33.790568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1ed6-7388-4490-85c9-4b882c7980e8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:33.791105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1ed6-7388-4490-85c9-4b882c7980e8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:33.791105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1ed6-7388-4490-85c9-4b882c7980e8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:33.792428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 635/1274] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:33.798443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:33.798910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:33.799195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:33.818266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:33.819471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:33.884499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 640/1275] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:33 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:33.908092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1ed6-7388-4490-85c9-4b882c7980e8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.006412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1ed6-7388-4490-85c9-4b882c7980e8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.023477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:34.040052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.040785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.041007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.041282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.051229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:17:34.056379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:34.057479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:34.058563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 636/1276] 10.222.0.144 () {56 vars in 1101 bytes} [Tue Jul 26 16:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:34.064093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.144049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:34.161409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01402d4-127d-45ac-bd78-64af640b7a90 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.195337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.195947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.196236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.196544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.197250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:34.197556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:34.198914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:34.207262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 641/1277] 10.222.0.144 () {56 vars in 1104 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:34.213061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.314362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc9c41-07a6-4795-9341-795a3151e189 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.347311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.347938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.348220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.348521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.358332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:34.359771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:34.375150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:34.379379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:34.380500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:34.380500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:34.380500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:34.380500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:34.380500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 637/1278] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:34.382811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.479834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289335-7c4d-4226-99db-a1acc4794312 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.516594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.517205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.517482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.517755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.518437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:34.518738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:34.520090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:34.531737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 642/1279] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:34.537519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.569240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:34.637096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b6f9e-4b92-4b29-831e-1d4738ea2ca7 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.669889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.670518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.670798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.671071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.678928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:34.680636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:17:34.691703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:34.692718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:34.693674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 638/1280] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:34.699098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.702906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:34.806914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb119ea-ea19-4a44-9abb-fe451116cec5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:34.839826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:34.840374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:34.840686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:34.840961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:34.841751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:34.842060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:34.843490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:34.861166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 643/1281] 10.222.0.144 () {56 vars in 1148 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:34.867121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:34.956738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:34.966044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d40c2b-28a2-4ed4-bfb0-ad4798bdc70e None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.000289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.000948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.001236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.001513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.066468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.067915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.153583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:35.163415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 639/1282] 10.222.0.144 () {58 vars in 1287 bytes} [Tue Jul 26 16:17:34 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:17:35.169563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:35.228849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:35.265670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6c09c-1e9d-44df-a649-1a79f3c2753f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.298828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.299456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.299735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.300008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.310035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.311465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:17:35.326435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:35.327452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:35.328550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:17:35.328550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:17:35.328550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:17:35.328550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:35.328550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 644/1283] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:35.333478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:35.430652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6a6db-e04e-45fb-b176-5796905e9e10 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.464893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.465547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.466162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.466350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.467039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:35.467345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.468731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.480018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 640/1284] 10.222.0.144 () {56 vars in 1114 bytes} [Tue Jul 26 16:17:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:17:35.485958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:35.586154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c565852-88e8-45af-8635-caff529b5c84 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.599965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:35.619554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.620160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.620473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.620746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.627621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.629093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:17:35.640866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:35.642269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 26 16:17:35.643143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 645/1285] 10.222.0.144 () {56 vars in 1116 bytes} [Tue Jul 26 16:17:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:35.648965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:35.699266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:35.747021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56c874-aedf-4099-b21c-983c43eaeb69 None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.781161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.781885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.782391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.782771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.783683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:35.784027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.785847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849124d-4eac-4374-a629-7d9fdaca71c3 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.799919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 641/1286] 10.222.0.144 () {56 vars in 1153 bytes} [Tue Jul 26 16:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:17:35.806462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:35.907133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c197-ce17-47d8-8c3a-e4c5d3c6ae53 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:35.941121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:35.941719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:35.941999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:35.942271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:35.943048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] RBAC: Adding query filter params (role.id=96e631f147a74de7b904e1ecc4b4d8f8, scope.project.id=dc753d253044415fbf36fa712bea0602, user.id=8c7f8c220a3844469072a97f7d2cb6e2) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:35.943350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:35.944802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57974fb6-b1d2-4594-bd25-77c3983ded40 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:35.950432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 646/1287] 10.222.0.144 () {56 vars in 1379 bytes} [Tue Jul 26 16:17:35 2022] GET /identity/v3/role_assignments?user.id=8c7f8c220a3844469072a97f7d2cb6e2&scope.project.id=dc753d253044415fbf36fa712bea0602&role.id=96e631f147a74de7b904e1ecc4b4d8f8 => generated 610 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:17:37.987100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082357-2c98-4653-94d3-6ac63a3d3f4f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:37.987657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082357-2c98-4653-94d3-6ac63a3d3f4f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:37.987657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae082357-2c98-4653-94d3-6ac63a3d3f4f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:37.989052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 642/1288] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:37.994044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:37.994487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:37.994582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:38.012677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:38.013843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:38.042361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:38.078206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 647/1289] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:37 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:38.099685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae082357-2c98-4653-94d3-6ac63a3d3f4f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:38.203000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae082357-2c98-4653-94d3-6ac63a3d3f4f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:38.213208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:38.235046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:38.236214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:38.236517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:38.236789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:38.237450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:38.238748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:38.247384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:38.248648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:38.249524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:38.250277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 643/1290] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:17:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:38.255061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:38.350857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b017e-39fe-4651-83fd-ccce23d28809 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:38.384472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:38.385077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:38.385361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:38.385634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:38.386298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:38.386603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:38.388011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b53e1-e6bd-42cb-ac95-3f15c9602792 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:38.392726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 648/1291] 10.222.0.144 () {56 vars in 1121 bytes} [Tue Jul 26 16:17:38 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:38.398473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:38.498642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0e0c6-24b8-4744-ba6f-5bdfe9ece49f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:38.534255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:38.534859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:38.535138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:38.535407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] PATH_INFO: `/v3/services` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:38.536074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:38.536375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:38.537754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed31cd-41d4-4294-ab47-e47f722c75a2 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:38.542095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 644/1292] 10.222.0.144 () {56 vars in 1121 bytes} [Tue Jul 26 16:17:38 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:40.538788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc094-2165-48c4-b79e-c0f56a8080c4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:40.539345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc094-2165-48c4-b79e-c0f56a8080c4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:40.539436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc094-2165-48c4-b79e-c0f56a8080c4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:40.540828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 649/1293] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:40.546482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c989-67b9-4394-a470-52a848b81c37 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:40.547014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c989-67b9-4394-a470-52a848b81c37 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:40.547014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c989-67b9-4394-a470-52a848b81c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:40.565156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1366c989-67b9-4394-a470-52a848b81c37 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:40.566340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1366c989-67b9-4394-a470-52a848b81c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:40.596250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:40.631740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 645/1294] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:40 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:40.654677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc094-2165-48c4-b79e-c0f56a8080c4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:40.685978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:40.755971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc094-2165-48c4-b79e-c0f56a8080c4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:40.789161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:40.789771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:40.790053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:40.790324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:40.791005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:40.792362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9fd62-a14c-490f-a7cf-c39c61106155 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:40.892010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 650/1295] 10.222.0.144 () {60 vars in 1146 bytes} [Tue Jul 26 16:17:40 2022] POST /identity/v3/services => generated 247 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:17:42.894253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cc438-e286-4a63-81d1-6bdb8bf3ddd9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:42.894803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cc438-e286-4a63-81d1-6bdb8bf3ddd9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:42.894803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cc438-e286-4a63-81d1-6bdb8bf3ddd9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:42.896146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 646/1296] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:42.901315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:42.901692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:42.901866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:42.920437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:42.921695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:42.991634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 651/1297] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:42 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:43.014123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cc438-e286-4a63-81d1-6bdb8bf3ddd9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:43.111250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cc438-e286-4a63-81d1-6bdb8bf3ddd9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:43.145756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:43.146416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:43.146698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:43.146967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:43.147665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:43.149114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:43.158673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:43.159854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:43.160857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 647/1298] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:17:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:43.166017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:43.184518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:43.265285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944dbbe-fa4c-415f-b656-3ae8f166fb4d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:43.299267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:43.299915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:43.300892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:43.301159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:43.301822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:43.302123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:43.303478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:43.308759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 652/1299] 10.222.0.144 () {56 vars in 1121 bytes} [Tue Jul 26 16:17:43 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:43.315355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:43.346208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:43.413211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4e2fd-c8ff-41b6-903b-f87c6fc28f6d None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:43.448036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:43.448675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:43.448961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:43.449235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:43.449992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] RBAC: Adding query filter params (interface=public, service_id=cc161132fde84b8f82e63701ba7e9536, region_id=RegionOne) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:43.450304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:43.451694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08a38-c76d-403e-8d5a-f53b4fb99e00 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:43.456734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 648/1300] 10.222.0.144 () {56 vars in 1256 bytes} [Tue Jul 26 16:17:43 2022] GET /identity/v3/endpoints?service_id=cc161132fde84b8f82e63701ba7e9536&interface=public®ion_id=RegionOne => generated 195 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:43.462395 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:43.548323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:43.568339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce7745-4914-417d-ad87-1d916ee0fb86 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:43.602299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:43.602961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:43.603240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:43.603509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:43.604145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:43.605535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bd39d-62fb-4c8d-9878-23213e74b7c8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:43.612618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 653/1301] 10.222.0.144 () {56 vars in 1092 bytes} [Tue Jul 26 16:17:43 2022] GET /identity/v3/services => generated 2201 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:17:45.597986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd8e03-aa29-430e-878b-37b0f2af9583 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:45.598507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd8e03-aa29-430e-878b-37b0f2af9583 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:45.598619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd8e03-aa29-430e-878b-37b0f2af9583 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:45.599898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 649/1302] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:17:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:17:45.605037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:45.605461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:45.605588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:45.623324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:17:45.624897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:17:45.690680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 654/1303] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:17:45 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:17:45.713048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd8e03-aa29-430e-878b-37b0f2af9583 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:45.805813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:45.812316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd8e03-aa29-430e-878b-37b0f2af9583 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:45.844678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:45.845303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:45.846197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:45.846477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:45.847095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:45.848466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:17:45.857088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:17:45.858302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application with Lock( Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: ERROR keystone.server.flask.application Jul 26 16:17:45.859158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 650/1304] 10.222.0.144 () {56 vars in 1122 bytes} [Tue Jul 26 16:17:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:17:45.864235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:45.938561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:45.961149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f68f5-126f-45f3-b9df-ef1b1c0bfa21 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:45.994595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:45.995198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:45.995475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:45.995746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:45.996547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:17:45.996953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:45.998302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e593-73bf-4df9-aada-7c9504f011cf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:46.003271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 655/1305] 10.222.0.144 () {56 vars in 1121 bytes} [Tue Jul 26 16:17:45 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:17:46.009210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:17:46.106270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858787e-7d2d-45f7-8d82-475febc62f9f None admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:17:46.138443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:17:46.139147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:17:46.139430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:17:46.139697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:17:46.140404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:17:46.141756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d7cea-272f-4bb0-a952-1d54934ad481 None admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:17:46.221386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 651/1306] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:17:46 2022] POST /identity/v3/endpoints => generated 333 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:17:56.000272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=30 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:17:56.216006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:23:53.194425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01428b1-327d-44e5-a003-dfcc196860ad None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:53.195363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01428b1-327d-44e5-a003-dfcc196860ad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:53.195789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01428b1-327d-44e5-a003-dfcc196860ad None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:53.203237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230a97-36dd-40d9-a677-2331c1eb8ef7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:53.204133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230a97-36dd-40d9-a677-2331c1eb8ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:53.204590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230a97-36dd-40d9-a677-2331c1eb8ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:53.210174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 656/1307] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:23:53 2022] GET /identity => generated 271 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:23:53.295851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0f230a97-36dd-40d9-a677-2331c1eb8ef7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:23:53.297351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0f230a97-36dd-40d9-a677-2331c1eb8ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:53.424876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 652/1308] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:23:53.476616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148fff-4883-472e-99c0-58e6da93acee None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:53.477139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148fff-4883-472e-99c0-58e6da93acee None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:53.477582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148fff-4883-472e-99c0-58e6da93acee None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:53.479327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 657/1309] 10.222.0.144 () {54 vars in 970 bytes} [Tue Jul 26 16:23:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:23:53.486177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83066ce1-fd1c-44f7-8cfd-c83d6ebb7d2f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:53.486635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83066ce1-fd1c-44f7-8cfd-c83d6ebb7d2f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:53.487176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83066ce1-fd1c-44f7-8cfd-c83d6ebb7d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:53.681904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-83066ce1-fd1c-44f7-8cfd-c83d6ebb7d2f None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:23:53.681904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-83066ce1-fd1c-44f7-8cfd-c83d6ebb7d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:53.720191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:23:53.779000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 653/1310] 10.222.0.144 () {58 vars in 1067 bytes} [Tue Jul 26 16:23:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:23:53.788416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14148fff-4883-472e-99c0-58e6da93acee None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:23:53.790996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-14148fff-4883-472e-99c0-58e6da93acee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:53.910669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14148fff-4883-472e-99c0-58e6da93acee None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:23:53.987580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:23:53.988092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:53.988378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:53.988684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:53.990332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:54.049621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:23:54.107910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:23:54.109377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc817f28-a627-4d22-a152-c5bd71093d9e service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:23:54.129023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 658/1311] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:23:56.461260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:56.469656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:56.469656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:56.469656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 654/1312] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:23:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:23:56.469927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d0cf-5ef2-4c68-8b62-b4973a3e0e9b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:56.469927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d0cf-5ef2-4c68-8b62-b4973a3e0e9b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:56.469927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d0cf-5ef2-4c68-8b62-b4973a3e0e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:56.548429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4474d0cf-5ef2-4c68-8b62-b4973a3e0e9b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:23:56.549692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4474d0cf-5ef2-4c68-8b62-b4973a3e0e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:56.651363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 659/1313] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:23:56.689192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:23:56.690915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:56.837523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d284c-199c-4def-9cdc-792c8638e25c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:23:56.912056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:23:56.915087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:23:56.915719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:56.916004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:56.916280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:56.918006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:57.029241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:23:57.030665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559380f9-5121-43d3-8098-caac63cf525b service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:23:57.048346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 655/1314] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:23:59.695804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:59.696361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:59.696361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:59.697749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 660/1315] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:23:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:23:59.702833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca91f-b8b1-4211-8a18-5a4e0477a154 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:23:59.703209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca91f-b8b1-4211-8a18-5a4e0477a154 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:23:59.703414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca91f-b8b1-4211-8a18-5a4e0477a154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:23:59.769123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0c9ca91f-b8b1-4211-8a18-5a4e0477a154 None None] MFA Rules not processed for user `7806a5c324534a3fa0c5c442f35fb401`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:23:59.770277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0c9ca91f-b8b1-4211-8a18-5a4e0477a154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:59.821811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:23:59.865988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 656/1316] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:23:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:23:59.904979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:23:59.906507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:23:59.916716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:00.008516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba343ce-49e4-46c4-be9a-acbbb4c33114 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:00.083042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:00.083042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:00.083042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:00.083042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:00.084013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:00.144538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:00.201143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:00.203014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98be37-2b81-4fa6-9c00-3e6836a2f20a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:00.224674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 661/1317] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:23:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:03.141603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edab1-3620-49a6-b7cc-53deb1fedfe4 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:03.142173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edab1-3620-49a6-b7cc-53deb1fedfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:03.142173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edab1-3620-49a6-b7cc-53deb1fedfe4 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:03.143522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 657/1318] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:03.149172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd7f4-1ef1-4af0-bd82-702a5d2cb96b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:03.149622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd7f4-1ef1-4af0-bd82-702a5d2cb96b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:03.149759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd7f4-1ef1-4af0-bd82-702a5d2cb96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:03.215271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-46bdd7f4-1ef1-4af0-bd82-702a5d2cb96b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:03.216533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-46bdd7f4-1ef1-4af0-bd82-702a5d2cb96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:03.310284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 662/1319] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:03.362222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:03.362509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:03.362789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:03.364148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 658/1320] 10.222.0.144 () {54 vars in 970 bytes} [Tue Jul 26 16:24:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:03.370315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727318-52b9-4a8b-9641-cbd1cc86dcb6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:03.370605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727318-52b9-4a8b-9641-cbd1cc86dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:03.370878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727318-52b9-4a8b-9641-cbd1cc86dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:03.392450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:03.445011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f727318-52b9-4a8b-9641-cbd1cc86dcb6 None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:03.446211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f727318-52b9-4a8b-9641-cbd1cc86dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:03.520690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 663/1321] 10.222.0.144 () {58 vars in 1067 bytes} [Tue Jul 26 16:24:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:03.530372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:03.531896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:03.634871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93bdbe-46de-46a3-92f5-8fba64fb5c64 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:03.711510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:03.712047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:03.712319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:03.712685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:03.714686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:03.723960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:03.827270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:03.828746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ec986-0cfd-484d-a59e-5a1116f346b4 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:03.847025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 659/1322] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:24:07.266077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:07.266636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:07.266636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:07.267977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 664/1323] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:24:07.273798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c55-0494-49dc-a0c1-c01a16cad485 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:07.274252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c55-0494-49dc-a0c1-c01a16cad485 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:07.274346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb53c55-0494-49dc-a0c1-c01a16cad485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:07.339141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5cb53c55-0494-49dc-a0c1-c01a16cad485 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:07.340344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5cb53c55-0494-49dc-a0c1-c01a16cad485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:07.397674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:07.437136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 660/1324] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:07.473287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:07.475391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:07.584629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f6fd-143b-4a5f-9056-da96d2443d64 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:07.598789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:07.667825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:07.668475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:07.668756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:07.669027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:07.670704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:07.787345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:07.788826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9fb8-ebab-410f-92e3-81c753bbd25a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:07.806153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 665/1325] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:09.885761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:09.886293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:09.886414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:09.887805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 661/1326] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:09.893141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e8dc-acdc-4ef1-ab0f-b12475ec8f85 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:09.893550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e8dc-acdc-4ef1-ab0f-b12475ec8f85 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:09.893704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e8dc-acdc-4ef1-ab0f-b12475ec8f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:09.943510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:09.962122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c605e8dc-acdc-4ef1-ab0f-b12475ec8f85 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:09.963354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c605e8dc-acdc-4ef1-ab0f-b12475ec8f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:10.057782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 666/1327] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:10.094434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:10.095954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:10.199104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415432-42ad-4191-86c2-87c703fca5a6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:10.277619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:10.278238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:10.278518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:10.278792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:10.280438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:10.301534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:10.394075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:10.395510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e076-643d-4eb5-be37-e8ca2f514220 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:10.413512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 662/1328] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:12.493020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:12.493020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:12.493020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:12.494179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 667/1329] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:12.499410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ac093-a5b4-465f-b879-fc31e9f38ff5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:12.499826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ac093-a5b4-465f-b879-fc31e9f38ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:12.499974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ac093-a5b4-465f-b879-fc31e9f38ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:12.566132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-238ac093-a5b4-465f-b879-fc31e9f38ff5 None None] MFA Rules not processed for user `7806a5c324534a3fa0c5c442f35fb401`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:12.567355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-238ac093-a5b4-465f-b879-fc31e9f38ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:12.635200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:12.667725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 663/1330] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:12.704345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:12.705921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:12.812130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3a8f-bb52-4f43-adcf-3625802185c5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:12.832991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:12.889402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:12.890035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:12.890315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:12.890588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:12.892299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:13.002622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:13.004059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbcf11-6248-4f1f-92ac-e10418dc5405 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:13.023707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 668/1331] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:15.483319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:15.483846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:15.483846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:15.485246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 664/1332] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:15.490840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47956-6a6e-4414-a716-8daae90e87d4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:15.491289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47956-6a6e-4414-a716-8daae90e87d4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:15.491383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47956-6a6e-4414-a716-8daae90e87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:15.553957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:15.556603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ead47956-6a6e-4414-a716-8daae90e87d4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:15.557801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ead47956-6a6e-4414-a716-8daae90e87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:15.651181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 669/1333] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:15.689844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:15.691450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:15.792333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff51ee0-c9e3-4d71-ad49-ae705f973ad2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:15.863550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:15.864157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:15.864463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:15.864736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:15.867052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:15.892858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:15.971938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:15.973301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c19279-9e68-47ce-b707-16956192bbb5 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:15.993869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 665/1334] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:24:18.015988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:18.016573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:18.016573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:18.017912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 670/1335] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:18.023902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceb48b-72a0-4dae-b9d9-15bb70da241e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:18.024310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceb48b-72a0-4dae-b9d9-15bb70da241e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:18.024480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceb48b-72a0-4dae-b9d9-15bb70da241e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:18.093354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-05ceb48b-72a0-4dae-b9d9-15bb70da241e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:18.094572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-05ceb48b-72a0-4dae-b9d9-15bb70da241e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:18.167315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:18.188932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 666/1336] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:24:18.229729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:18.231261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:18.331317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484a6c7-8450-4db2-8faa-a5fc7e8b9ab4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:18.358655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:18.407123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:18.407741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:18.408021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:18.408294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:18.409945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:18.517969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:18.519448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827f856-9a5a-44ec-84c2-56d16e984ff0 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:18.538502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 671/1337] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:21.173628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:21.174193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:21.174193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:21.175585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 667/1338] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:21.181192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105a70-6e0a-48c4-ac91-27188366e028 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:21.181605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105a70-6e0a-48c4-ac91-27188366e028 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:21.181732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105a70-6e0a-48c4-ac91-27188366e028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:21.245671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0a105a70-6e0a-48c4-ac91-27188366e028 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:21.246894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0a105a70-6e0a-48c4-ac91-27188366e028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:21.255121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:21.340957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 672/1339] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:21.377334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:21.378893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:21.479463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8ae7f-e08e-4d0e-ae1a-669764f489ef None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:21.554389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:21.554994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:21.555270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:21.555538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:21.557167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:21.594036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:21.665369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:21.666784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c939-6190-445d-8d5b-ce4a704a2466 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:21.685391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 668/1340] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:25.878003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:25.878539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:25.878539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:25.880165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 673/1341] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:25.885396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e336a7-4615-473e-b55c-0b59916be02b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:25.885889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e336a7-4615-473e-b55c-0b59916be02b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:25.886212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e336a7-4615-473e-b55c-0b59916be02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:25.962250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-68e336a7-4615-473e-b55c-0b59916be02b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:25.963474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-68e336a7-4615-473e-b55c-0b59916be02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:26.058609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:26.070280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 669/1342] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:26.108654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:26.110209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:26.217110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0382-9870-4967-823f-9fe99f8af8f2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:26.254759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:26.291599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:26.292210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:26.292531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:26.292810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:26.294426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:26.403464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:26.404940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cc23-18d4-4d42-96ff-66aff6f1dd1d service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:26.423791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 674/1343] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:29.397502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:29.398061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:29.398061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:29.399444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 670/1344] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:29.404758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7b41-932a-496e-8c49-6b0d8aa8cfe6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:29.405167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7b41-932a-496e-8c49-6b0d8aa8cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:29.405434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7b41-932a-496e-8c49-6b0d8aa8cfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:29.474484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5bec7b41-932a-496e-8c49-6b0d8aa8cfe6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:29.475944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5bec7b41-932a-496e-8c49-6b0d8aa8cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:29.498615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:29.581367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 675/1345] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:29.620101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:29.621776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:29.726732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c412d28-733e-4ba2-910b-6eed4780ccd7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:29.799970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:29.800688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:29.800970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:29.801245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:29.802871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:29.846932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:29.921120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:29.922706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68acf-b2c3-4293-9962-4ec3042ff3f5 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:29.942001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 671/1346] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:24:34.192290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:34.192892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:34.192892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:34.194225 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 676/1347] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:34.199903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586319-4b4d-4436-aa18-b860582b3bb2 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:34.200421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586319-4b4d-4436-aa18-b860582b3bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:34.200507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7586319-4b4d-4436-aa18-b860582b3bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:34.273746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d7586319-4b4d-4436-aa18-b860582b3bb2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:34.274977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d7586319-4b4d-4436-aa18-b860582b3bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:34.373885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:34.376453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 672/1348] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:34.415719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:34.417329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:34.526138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b84b8-3815-4bbc-904b-75ddde613bc5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:34.576332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:34.607797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:34.608446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:34.608730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:34.608996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:34.610679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:34.724528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:34.725975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab567d5-6fd0-4871-8757-bf420ea9a660 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:34.744922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 677/1349] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:39.698592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b78cf-9228-4af8-9d7a-8b0f4b4ab0a5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:39.699079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b78cf-9228-4af8-9d7a-8b0f4b4ab0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:39.699079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b78cf-9228-4af8-9d7a-8b0f4b4ab0a5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:39.700522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 673/1350] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:24:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:39.705803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c4ac9-4463-46e7-8832-80309364a74e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:39.706228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c4ac9-4463-46e7-8832-80309364a74e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:39.706374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c4ac9-4463-46e7-8832-80309364a74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:39.846008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c31c4ac9-4463-46e7-8832-80309364a74e None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:39.847262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c31c4ac9-4463-46e7-8832-80309364a74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:39.866208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:39.966723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 678/1351] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:44.081574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43d74-fba3-4496-bc56-93cd6ca04686 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:44.082092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43d74-fba3-4496-bc56-93cd6ca04686 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:44.082092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c43d74-fba3-4496-bc56-93cd6ca04686 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:44.083662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 674/1352] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:24:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:44.088721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e633-6f67-4c87-a79c-af84096f4313 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:44.089155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e633-6f67-4c87-a79c-af84096f4313 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:44.089258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e633-6f67-4c87-a79c-af84096f4313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:44.170194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7d67e633-6f67-4c87-a79c-af84096f4313 None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:44.171395 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d67e633-6f67-4c87-a79c-af84096f4313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:44.244246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:44.281371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 679/1353] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:45.800700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860803b-982e-4061-b3e1-bf9a12113a7b None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:45.801245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860803b-982e-4061-b3e1-bf9a12113a7b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:45.801245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860803b-982e-4061-b3e1-bf9a12113a7b None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:45.802530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 675/1354] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:24:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:45.807590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5649047-a40f-4298-87c9-fd4080f87367 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:45.808038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5649047-a40f-4298-87c9-fd4080f87367 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:45.808232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5649047-a40f-4298-87c9-fd4080f87367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:45.885462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b5649047-a40f-4298-87c9-fd4080f87367 None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:45.886675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b5649047-a40f-4298-87c9-fd4080f87367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:46.008361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 680/1355] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:49.295611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf90a-f804-4cab-8f0c-65c7b5553ff7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:49.296119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf90a-f804-4cab-8f0c-65c7b5553ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:49.296119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf90a-f804-4cab-8f0c-65c7b5553ff7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:49.298961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 676/1356] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:49.305158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb0f8-a76a-471d-8203-3522728b2251 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:49.306033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb0f8-a76a-471d-8203-3522728b2251 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:49.306311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb0f8-a76a-471d-8203-3522728b2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:49.349094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:49.390731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-87acb0f8-a76a-471d-8203-3522728b2251 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:49.392094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-87acb0f8-a76a-471d-8203-3522728b2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:49.504533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 681/1357] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:49.559282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:49.559562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:49.560004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:49.561677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 677/1358] 10.222.0.144 () {54 vars in 967 bytes} [Tue Jul 26 16:24:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:49.572280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f36-8355-4a3f-abb7-2817ec1a56da None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:49.572590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f36-8355-4a3f-abb7-2817ec1a56da None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:49.572855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f36-8355-4a3f-abb7-2817ec1a56da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:49.655784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b85e7f36-8355-4a3f-abb7-2817ec1a56da None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:49.657484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b85e7f36-8355-4a3f-abb7-2817ec1a56da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:49.713906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:49.786437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 682/1359] 10.222.0.144 () {58 vars in 1064 bytes} [Tue Jul 26 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:49.794767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:49.796317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:49.960417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7d5b-560d-4cb4-8f7f-0c1a4a42a0eb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:50.039910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:50.082182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:50.082805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:50.083085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:50.083359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:50.085196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:50.177714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:50.179859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129e1b4-ff22-4ab7-b568-4c568dae9ed6 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:50.199885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 678/1360] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:52.437890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:52.438383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:52.438497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:52.439929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 683/1361] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:52.445578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7a523-b61b-46c8-b169-6c001ac7baf5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:52.446055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7a523-b61b-46c8-b169-6c001ac7baf5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:52.446138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7a523-b61b-46c8-b169-6c001ac7baf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:52.524131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-06f7a523-b61b-46c8-b169-6c001ac7baf5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:52.525379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-06f7a523-b61b-46c8-b169-6c001ac7baf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:52.564735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:52.625581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 679/1362] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:52.671223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:52.672777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:52.850977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b47f5-3d3f-4b63-91b9-60909fced763 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:52.861131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:52.983499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:52.984126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:52.984126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:52.984664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:52.986583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:53.090892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:53.092533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f97883-891d-41ef-9a09-50cc6f7fa96f service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:53.111164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 684/1363] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:55.615497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44d54-2aa4-4367-8cd7-fa251d6fdfec None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:55.616080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44d54-2aa4-4367-8cd7-fa251d6fdfec None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:55.616080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44d54-2aa4-4367-8cd7-fa251d6fdfec None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:55.618015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 680/1364] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:55.627659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24798-412c-41fb-af42-5023337d68bf None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:55.628201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24798-412c-41fb-af42-5023337d68bf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:55.628201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24798-412c-41fb-af42-5023337d68bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:55.645227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:55.702931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ead24798-412c-41fb-af42-5023337d68bf None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:55.704216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ead24798-412c-41fb-af42-5023337d68bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:55.808970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 685/1365] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:55.854161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:55.854472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:55.854679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:55.856274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 681/1366] 10.222.0.144 () {54 vars in 967 bytes} [Tue Jul 26 16:24:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:55.862874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504e35-0ad1-43c5-b7f0-0060de1c412e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:55.868308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504e35-0ad1-43c5-b7f0-0060de1c412e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:55.868308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504e35-0ad1-43c5-b7f0-0060de1c412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:55.944086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3f504e35-0ad1-43c5-b7f0-0060de1c412e None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:55.945885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3f504e35-0ad1-43c5-b7f0-0060de1c412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:55.981638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:56.067514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 686/1367] 10.222.0.144 () {58 vars in 1064 bytes} [Tue Jul 26 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:56.075975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:56.077538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:56.240729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7650fa-100d-4656-97f9-7a8305c7adca None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:56.264169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:56.360406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:56.360956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:56.361243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:56.361519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:56.363143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:56.456087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:56.457718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67357c2-2f04-48e0-8fe4-bc4301158b9a service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:56.477913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 682/1368] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:24:57.459535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5b08-b90a-4da3-a5ae-5561311bc2b7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:57.460628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5b08-b90a-4da3-a5ae-5561311bc2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:57.460862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5b08-b90a-4da3-a5ae-5561311bc2b7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:57.462259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 687/1369] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:57.467583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c2043-0109-4b73-a8c7-11380201358b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:57.468032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c2043-0109-4b73-a8c7-11380201358b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:57.468126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c2043-0109-4b73-a8c7-11380201358b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:57.502699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:57.539055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b50c2043-0109-4b73-a8c7-11380201358b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:57.540364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b50c2043-0109-4b73-a8c7-11380201358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:57.644865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 683/1370] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:57.988278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:57.989113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:57.989563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:57.991176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 688/1371] 10.222.0.144 () {54 vars in 969 bytes} [Tue Jul 26 16:24:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:57.996875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87d75-43a7-407b-9a5c-b23bbcbc0c62 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:57.997171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87d75-43a7-407b-9a5c-b23bbcbc0c62 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:57.997452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87d75-43a7-407b-9a5c-b23bbcbc0c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:58.076358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-79a87d75-43a7-407b-9a5c-b23bbcbc0c62 None None] MFA Rules not processed for user `5361a7650f164ff8b11abfe6b3c4c382`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:58.077604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-79a87d75-43a7-407b-9a5c-b23bbcbc0c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:58.126890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:58.157563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 684/1372] 10.222.0.144 () {58 vars in 1066 bytes} [Tue Jul 26 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:58.165866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:58.167588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:58.277633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013f330-a662-44d3-8a8b-2c80979a9968 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:58.323818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:58.358381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:58.359137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:58.359268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:58.359540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:58.361202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:58.473459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:58.475066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17cdc-0d31-48e5-aa56-b98e4b40793b service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:58.495147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 689/1373] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:24:58.714251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:58.715293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:58.715293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:58.716380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 685/1374] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:24:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:24:58.721506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c728b1-e0e5-4b2e-a88f-4d865d5ae461 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:58.721999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c728b1-e0e5-4b2e-a88f-4d865d5ae461 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:58.722148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c728b1-e0e5-4b2e-a88f-4d865d5ae461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:58.790309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-77c728b1-e0e5-4b2e-a88f-4d865d5ae461 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:24:58.791531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-77c728b1-e0e5-4b2e-a88f-4d865d5ae461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:58.812438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:58.907567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 690/1375] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:24:58.947485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:24:58.949015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:59.112072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7553780-4288-49e6-8511-2acdc7a4eb0f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:24:59.163827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:24:59.226877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:24:59.230856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:24:59.230856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:24:59.230856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:24:59.230856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:24:59.321341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:24:59.323175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ef11e-fe3a-43a6-8f2c-3009e43685de service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:24:59.341899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 686/1376] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:01.499133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:01.499569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:01.503992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:01.503992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 691/1377] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:01.508360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ba9-fee9-471f-bd63-fa39ead9c5e7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:01.509226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ba9-fee9-471f-bd63-fa39ead9c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:01.509503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ba9-fee9-471f-bd63-fa39ead9c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:01.577994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-89243ba9-fee9-471f-bd63-fa39ead9c5e7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:01.579582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-89243ba9-fee9-471f-bd63-fa39ead9c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:01.590006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:01.732712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 687/1378] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:01.816146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:01.816146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:01.928762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6a9d-3866-498a-8872-f0c4835e448d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:01.932328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:02.061184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:02.063091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:02.063091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:02.063091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:02.065916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:02.183537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:02.185552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851d4c9-7407-4ca2-b26d-577e7aa9726a service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:02.215198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 692/1379] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:04.429633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:04.430198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:04.430198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:04.431709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 688/1380] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:04.436750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8a2ac-9e49-410a-8d2f-9b5535f2899d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:04.437179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8a2ac-9e49-410a-8d2f-9b5535f2899d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:04.437358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8a2ac-9e49-410a-8d2f-9b5535f2899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:04.448818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:04.513489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8bd8a2ac-9e49-410a-8d2f-9b5535f2899d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:04.514706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8bd8a2ac-9e49-410a-8d2f-9b5535f2899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:04.630125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 693/1381] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:04.674913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:04.676600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:04.823303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:04.834518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facdff-61fe-4078-85ba-a1020bf9da4a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:04.953339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:04.954344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:04.954727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:04.955101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:04.956856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:05.063839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:05.065711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9516920-6c48-40a6-ab5c-834ab370c41f service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:05.079360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:05.089096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 689/1382] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:25:07.528806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:07.529357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:07.529465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:07.531155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 694/1383] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:07.537651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6820-563e-4d30-94c8-cd74989d9a9c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:07.538148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6820-563e-4d30-94c8-cd74989d9a9c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:07.538326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6820-563e-4d30-94c8-cd74989d9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:07.608875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-325c6820-563e-4d30-94c8-cd74989d9a9c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:07.610190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-325c6820-563e-4d30-94c8-cd74989d9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:07.729966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 690/1384] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:07.780318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:07.781798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:07.902817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:07.976468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1b8e4-910a-461c-a247-d610749a6c16 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:08.106619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:08.107346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:08.107634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:08.107909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:08.109583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:08.165546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:08.209940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:08.212124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cad54d-dbcc-47b0-aebe-2eaa53330844 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:08.234290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 695/1385] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:10.536026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:10.536607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:10.536802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:10.538542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 691/1386] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:10.544952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8de8-8398-47e1-8c13-6425df670a94 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:10.545394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8de8-8398-47e1-8c13-6425df670a94 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:10.545590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8de8-8398-47e1-8c13-6425df670a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:10.621091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1f3d8de8-8398-47e1-8c13-6425df670a94 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:10.622387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1f3d8de8-8398-47e1-8c13-6425df670a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:10.720510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 696/1387] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:10.777545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:10.779194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:10.871156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:10.942603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ed826-37dd-4206-a200-b459ac07c51a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:11.066239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:11.066771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:11.067062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:11.067332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:11.068996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:11.121356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:11.181266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:11.182865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7b904-39b7-4361-acaa-a0ed51f3c416 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:11.201973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 692/1388] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:13.431220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:13.431838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:13.431838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:13.433133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 697/1389] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:13.438345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b659-ed41-469a-aa66-da9e43ea11db None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:13.438762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b659-ed41-469a-aa66-da9e43ea11db None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:13.438899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b659-ed41-469a-aa66-da9e43ea11db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:13.513601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4895b659-ed41-469a-aa66-da9e43ea11db None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:13.514732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4895b659-ed41-469a-aa66-da9e43ea11db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:13.609950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 693/1390] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:13.654463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:13.656051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:13.687052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:13.802156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa540ef1-4c82-4aa6-8be0-a55c34079870 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:13.907519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:13.919383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:13.927542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:13.927542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:13.927542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:13.927542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:13.937210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38e01-4798-4121-9b56-cf091edc0771 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:13.937523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38e01-4798-4121-9b56-cf091edc0771 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:13.937798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38e01-4798-4121-9b56-cf091edc0771 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:13.939277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 694/1391] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:25:13.944337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6216cb-3529-4e86-8b7a-c203596f471f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:13.944652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6216cb-3529-4e86-8b7a-c203596f471f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:13.944952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6216cb-3529-4e86-8b7a-c203596f471f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:13.960866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:14.013158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-cb6216cb-3529-4e86-8b7a-c203596f471f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:14.014649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cb6216cb-3529-4e86-8b7a-c203596f471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:14.015014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:14.016646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b59563-694f-4066-89cb-9d1d4aa001f3 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:14.034962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 698/1392] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:14.111603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 695/1393] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:25:14.168653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:14.168938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:14.169242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:14.170614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 699/1394] 10.222.0.144 () {54 vars in 971 bytes} [Tue Jul 26 16:25:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:14.175837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142745dc-bfcc-4222-862a-3e49345ccdad None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:14.180880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142745dc-bfcc-4222-862a-3e49345ccdad None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:14.180880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142745dc-bfcc-4222-862a-3e49345ccdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:14.263002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-142745dc-bfcc-4222-862a-3e49345ccdad None None] MFA Rules not processed for user `b6d652014d6446a0ae162120aac0ed94`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:14.264257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-142745dc-bfcc-4222-862a-3e49345ccdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:14.303670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:14.348515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 696/1395] 10.222.0.144 () {58 vars in 1068 bytes} [Tue Jul 26 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:14.357176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:14.358704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:14.462513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc7efd-418c-4a86-af31-13dd5d949a8f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:14.476306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:14.537469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:14.538116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:14.538433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:14.538705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:14.540331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:14.650805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:14.652890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528553-9a7a-4c59-9bfe-26e060e8e090 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:14.671496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 700/1396] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:15.487727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a00c-6c98-470c-b958-d27601afe93c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:15.488196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a00c-6c98-470c-b958-d27601afe93c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:15.488622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a00c-6c98-470c-b958-d27601afe93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:15.682704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1381a00c-6c98-470c-b958-d27601afe93c None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:15.683950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1381a00c-6c98-470c-b958-d27601afe93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:15.778807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 697/1397] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:15.823029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:15.823537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:15.823641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:15.825023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 701/1398] 10.222.0.144 () {54 vars in 948 bytes} [Tue Jul 26 16:25:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:15.831648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad07df6-949a-403b-a0f5-8bcdd90bdf2f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:15.832018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad07df6-949a-403b-a0f5-8bcdd90bdf2f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:15.832357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad07df6-949a-403b-a0f5-8bcdd90bdf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:15.894825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:15.914195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5ad07df6-949a-403b-a0f5-8bcdd90bdf2f None None] MFA Rules not processed for user `24f380d935524a2295fd9c01a8fec95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:15.915403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5ad07df6-949a-403b-a0f5-8bcdd90bdf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.026650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 698/1399] 10.222.0.144 () {58 vars in 1045 bytes} [Tue Jul 26 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:16.036021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:16.037836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.083621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:16.174683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea19306-3137-49ce-a0e9-67c959c78c17 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:16.179872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d813a3-44bf-4a06-8db1-c9bd4917c1ef None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:16.180194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d813a3-44bf-4a06-8db1-c9bd4917c1ef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:16.180516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d813a3-44bf-4a06-8db1-c9bd4917c1ef None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:16.181873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 699/1400] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:25:16.186822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:16.187110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:16.187382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:16.254257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:16.255722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.280695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:16.281241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:16.281525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:16.281798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:16.283454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.312910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:16.314300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:16.354304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 700/1401] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:25:16.366111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:16.368235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec8b5e-3fed-4912-9ae5-c6cf721cb44c service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:16.376496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 702/1402] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:25:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:25:16.404137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:16.405372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.518156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3185897-eb74-4c2b-8356-b9efd954a4f8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:16.587123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:16.632600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:16.633211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:16.633488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:16.633757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:16.635369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:16.724167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:16.726287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f88663-3ff9-465a-a86b-e9ef45a9b3ee service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:16.744787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 701/1403] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:19.131850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:19.133315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:19.133315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:19.134332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 703/1404] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:19.140891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9260bd-ce35-485a-8e19-49a6f06d3892 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:19.143198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9260bd-ce35-485a-8e19-49a6f06d3892 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:19.143659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9260bd-ce35-485a-8e19-49a6f06d3892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:19.249099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2a9260bd-ce35-485a-8e19-49a6f06d3892 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:19.250340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2a9260bd-ce35-485a-8e19-49a6f06d3892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:19.289185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:19.372890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 702/1405] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:19.431437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:19.434144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:19.647651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346fa5f-0c54-405c-bc29-e87e848f2cd7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:19.676169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:19.818887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:19.819527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:19.820520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:19.820914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:19.822986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:19.928930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:19.930597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c42bd8-3e43-42c1-9b71-894ca0d3be64 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:19.954245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 704/1406] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:22.224323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:22.224914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:22.224914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:22.226282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 703/1407] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:25:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:22.232208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a5f17-7989-4944-873c-1d00b5133be0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:22.233909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a5f17-7989-4944-873c-1d00b5133be0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:22.234420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a5f17-7989-4944-873c-1d00b5133be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:22.276665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:22.326915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ca8a5f17-7989-4944-873c-1d00b5133be0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:22.328171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ca8a5f17-7989-4944-873c-1d00b5133be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:22.456428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 705/1408] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:22.517301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:22.520381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:22.730673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50161a97-7b41-482c-ad28-c0982992b65d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:22.753525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:22.869179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:22.869805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:22.870087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:22.870363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:22.872036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:22.982602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:22.984233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41859d1-a450-4415-8759-2ff85cbb55af service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:23.004619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 704/1409] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:32.454550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=53 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:33.002154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:40.812774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17be0-722a-4d15-bc13-0c1fbf99bfc4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:40.813340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17be0-722a-4d15-bc13-0c1fbf99bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:40.813340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a17be0-722a-4d15-bc13-0c1fbf99bfc4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:40.814603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 706/1410] 10.222.0.144 () {54 vars in 923 bytes} [Tue Jul 26 16:25:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:40.819763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179c8b-6b77-47ae-9339-39297aa8ef0b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:40.820201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179c8b-6b77-47ae-9339-39297aa8ef0b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:40.820304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179c8b-6b77-47ae-9339-39297aa8ef0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:40.893674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-05179c8b-6b77-47ae-9339-39297aa8ef0b None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:40.894886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-05179c8b-6b77-47ae-9339-39297aa8ef0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:40.998577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 705/1411] 10.222.0.144 () {58 vars in 1020 bytes} [Tue Jul 26 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:41.012897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80bfa3-0593-4389-9a2f-281eb077d8ca None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:41.013186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80bfa3-0593-4389-9a2f-281eb077d8ca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:41.013462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80bfa3-0593-4389-9a2f-281eb077d8ca None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:41.014837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 707/1412] 10.222.0.144 () {54 vars in 923 bytes} [Tue Jul 26 16:25:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:41.019207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9daaf-1d1a-4acf-9ab9-392ae33fb28e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:41.019514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9daaf-1d1a-4acf-9ab9-392ae33fb28e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:41.019784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9daaf-1d1a-4acf-9ab9-392ae33fb28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:41.090433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:41.093310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3fe9daaf-1d1a-4acf-9ab9-392ae33fb28e None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:41.094508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3fe9daaf-1d1a-4acf-9ab9-392ae33fb28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:41.207661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 706/1413] 10.222.0.144 () {58 vars in 1020 bytes} [Tue Jul 26 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:43.662910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:43.663456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:43.663456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:43.664864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:25:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:43.669980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0769e-52fb-42d5-a9d1-1af688b99b82 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:43.670415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0769e-52fb-42d5-a9d1-1af688b99b82 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:43.670524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0769e-52fb-42d5-a9d1-1af688b99b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:43.735660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b7c0769e-52fb-42d5-a9d1-1af688b99b82 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:43.736904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b7c0769e-52fb-42d5-a9d1-1af688b99b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:43.792060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:43.833404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:43.875498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:43.877105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:44.019919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b59348-df6a-4737-8d45-c5cb3c9f1597 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:44.097239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:44.140977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:44.141584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:44.141861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:44.142145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:44.143744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:44.229019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:44.230621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8bbe7-2d6b-4aa5-8ef1-c60059d48186 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:44.249737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 709/1416] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:45.146170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:45.146732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:45.146732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:45.148089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:45.153437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acca24-2f02-4efa-b1b8-161cebad62e1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:45.153861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acca24-2f02-4efa-b1b8-161cebad62e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:45.153988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acca24-2f02-4efa-b1b8-161cebad62e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:45.227557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-38acca24-2f02-4efa-b1b8-161cebad62e1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:45.228858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-38acca24-2f02-4efa-b1b8-161cebad62e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:45.255457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:45.324664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:45.474385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:45.475814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:45.622533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0876f-40ac-4a70-bd72-b644cbb79ea9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:45.663269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:45.734656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:45.735213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:45.735497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:45.735772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:45.737433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:45.824269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:45.825862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92458475-9ecd-4686-97f5-a7e99dcba1e7 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:45.843497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 709/1419] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:46.723330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:46.723937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:46.723937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:46.725272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:46.733174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3c1c-88b2-427d-8d00-03cfc246af2c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:46.733591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3c1c-88b2-427d-8d00-03cfc246af2c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:46.733724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3c1c-88b2-427d-8d00-03cfc246af2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:46.787181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:46.808819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e3af3c1c-88b2-427d-8d00-03cfc246af2c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:46.810022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e3af3c1c-88b2-427d-8d00-03cfc246af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:46.904094 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:47.058960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:47.060433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:47.203237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa95b1-d71e-4148-b6f3-1f502c3680d5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:47.215595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:47.315618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:47.316165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:47.316472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:47.316749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:47.318373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:47.405117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:47.406706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226d9e9-844e-4c12-9266-46cc5737f9b5 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:47.424691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 712/1422] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:48.289107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:48.290163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:48.290777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:48.292168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:48.297084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad4ef-29d7-4cdb-92d6-d45c21302e2b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:48.297563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad4ef-29d7-4cdb-92d6-d45c21302e2b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:48.297669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad4ef-29d7-4cdb-92d6-d45c21302e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:48.318148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:48.373508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d91ad4ef-29d7-4cdb-92d6-d45c21302e2b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:48.374724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d91ad4ef-29d7-4cdb-92d6-d45c21302e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:48.469602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:48.619701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:48.621287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:48.730274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:48.766164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fad76-bd5d-492f-9373-161e0456f559 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:48.873675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:48.874276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:48.874551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:48.874819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:48.876438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:48.949528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:48.963114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:48.964707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020416b-807f-4818-993f-fde097d2e372 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:48.982139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 712/1425] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:49.838285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:49.838861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:49.838861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:49.840179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:49.845078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17d599-f0f1-48b0-955a-0049145b5973 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:49.845506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17d599-f0f1-48b0-955a-0049145b5973 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:49.845638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17d599-f0f1-48b0-955a-0049145b5973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:49.921364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5a17d599-f0f1-48b0-955a-0049145b5973 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:49.922584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5a17d599-f0f1-48b0-955a-0049145b5973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:50.019692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:50.170403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:50.171946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:50.255141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:50.322576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86496c-5b6a-41a8-ab69-4c8a3ab6f750 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:50.440567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:50.441030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:50.441312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:50.441592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:50.443218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:50.491937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:50.536319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:50.537940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8780a0b-e597-42f8-bf1d-54c3f5f67d33 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:50.557008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 715/1428] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:51.415236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:51.415815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:51.415815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:51.417190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:51.421975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df01c8-2b96-4dc5-b703-d7e19218a134 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:51.422415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df01c8-2b96-4dc5-b703-d7e19218a134 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:51.422509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df01c8-2b96-4dc5-b703-d7e19218a134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:51.499769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-65df01c8-2b96-4dc5-b703-d7e19218a134 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:51.501025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-65df01c8-2b96-4dc5-b703-d7e19218a134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:51.602675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:51.744690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:51.746207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:51.795010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:51.890996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7bacc-a64b-4aa4-974a-796dafe90ed7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:52.001269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:52.001877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:52.002156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:52.002428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:52.004071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:52.011634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:52.093022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:52.094933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069cf0d-5266-417f-aa93-0f20e5ed7cf0 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:52.112672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 715/1431] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:52.984578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:52.985204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:52.985204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:52.986469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:52.991343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907a0d-853f-4a71-a0a4-e533efc5d8b8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:52.991794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907a0d-853f-4a71-a0a4-e533efc5d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:52.991901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79907a0d-853f-4a71-a0a4-e533efc5d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:53.066748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-79907a0d-853f-4a71-a0a4-e533efc5d8b8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:53.067950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-79907a0d-853f-4a71-a0a4-e533efc5d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:53.135208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:53.165504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:53.317889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:53.319430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:53.337099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:53.460206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a188e-edb1-47c7-b987-541ca3b9be5e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:53.553104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:53.579660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:53.580280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:53.580597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:53.580876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:53.582519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:53.681219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:53.682813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7402d4-122f-42a4-a786-38ca43790e44 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:53.700369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 718/1434] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:54.555930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:54.556483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:54.556483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:54.557918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:54.562884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2901-1af1-4bae-8721-a1fdd1879636 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:54.563311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2901-1af1-4bae-8721-a1fdd1879636 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:54.563474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2901-1af1-4bae-8721-a1fdd1879636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:54.639077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2c4e2901-1af1-4bae-8721-a1fdd1879636 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:54.640289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2c4e2901-1af1-4bae-8721-a1fdd1879636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:54.681305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:54.734292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:54.898000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:54.900298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:55.041310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe32a2-1f2f-4c96-9ce4-90d426cddcc2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:55.093102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:55.153163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:55.153765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:55.154047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:55.154321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:55.155953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:55.243753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:55.245359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36743b-91f5-4e50-9469-d49a9a1d29d4 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:55.263779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 718/1437] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:56.145507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:56.146059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:56.146059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:56.147413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:25:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:56.152998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978250f-bbb3-4a41-9c65-31a2c168058a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:56.153436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978250f-bbb3-4a41-9c65-31a2c168058a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:56.153556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978250f-bbb3-4a41-9c65-31a2c168058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:56.227823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3978250f-bbb3-4a41-9c65-31a2c168058a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:56.229717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3978250f-bbb3-4a41-9c65-31a2c168058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:56.231758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:56.326605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:56.486856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:56.488347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:56.632304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714869bb-7f60-48b5-9cd3-3749a6334203 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:56.659731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:56.744121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:56.744684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:56.744967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:56.745239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:56.746873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:56.834261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:56.835869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eafc58-90bd-4bb8-a4ce-2514f75510b7 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:56.853491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 721/1440] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:25:58.743894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:58.744454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:58.744454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:58.745746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:25:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:25:58.750954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b2b80-4a09-400a-b790-acbb41a1a430 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:58.751400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b2b80-4a09-400a-b790-acbb41a1a430 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:58.751518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b2b80-4a09-400a-b790-acbb41a1a430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:58.793485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:58.819725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-898b2b80-4a09-400a-b790-acbb41a1a430 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:25:58.821547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-898b2b80-4a09-400a-b790-acbb41a1a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:58.919346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:25:58.958342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:25:58.959884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:59.062054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31e088-62e7-4fe9-ab9f-358253548ac2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:25:59.092919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:25:59.133503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:25:59.134109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:25:59.134384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:25:59.134655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:25:59.136247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:25:59.242524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:25:59.243978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97101224-2aee-4767-a155-4622befaf404 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:25:59.262101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 721/1443] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:01.415697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:01.416247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:01.416247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:01.417637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:01.423221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396f17-3d7d-4e32-a40b-a567998eaba0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:01.423604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396f17-3d7d-4e32-a40b-a567998eaba0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:01.423792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396f17-3d7d-4e32-a40b-a567998eaba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:01.492637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c9396f17-3d7d-4e32-a40b-a567998eaba0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:01.494429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c9396f17-3d7d-4e32-a40b-a567998eaba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:01.507785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:01.592279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:01.630578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:01.632377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:01.746714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05e33-cb5a-4a07-919d-758948811150 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:01.758679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:01.820374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:01.821045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:01.821326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:01.821599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:01.823201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:01.930801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:01.932218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57038c-4089-44a0-9c01-07baa017c79c service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:01.950128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 724/1446] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:04.324352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:04.324960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:04.324960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:04.326284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:04.331423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cd3bc-c5b4-4ebc-a1ed-8399b8a0bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:04.331871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cd3bc-c5b4-4ebc-a1ed-8399b8a0bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:04.332022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cd3bc-c5b4-4ebc-a1ed-8399b8a0bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:04.379457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:04.397790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-355cd3bc-c5b4-4ebc-a1ed-8399b8a0bfe2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:04.399025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-355cd3bc-c5b4-4ebc-a1ed-8399b8a0bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:04.499481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:26:04.538423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:04.539988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:04.641891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12f8ef-42a3-4377-a63e-d94e37b627b9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:04.682108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:04.714838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:04.715457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:04.715741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:04.716018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:04.717690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:04.824115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:04.825585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ade3a-0187-46ed-ba0a-792d751136e0 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:04.843246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 724/1449] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:06.955809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:06.956364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:06.956364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:06.957733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:06.962863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12c06-3ab9-493a-a3db-7da99a64578e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:06.963267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12c06-3ab9-493a-a3db-7da99a64578e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:06.963441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12c06-3ab9-493a-a3db-7da99a64578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:07.027791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-11b12c06-3ab9-493a-a3db-7da99a64578e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:07.029007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-11b12c06-3ab9-493a-a3db-7da99a64578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:07.049669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:07.124256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:07.162881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:07.164440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:07.266555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eae55-e932-494b-92b8-14e057ed0bcb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:07.285850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:07.339093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:07.339700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:07.339978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:07.340249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:07.341890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:07.447580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:07.449040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc4bb-d524-465e-85be-98c55ee66f8b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:07.466258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 727/1452] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:26:09.529859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:09.530340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:09.530447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:09.531770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:09.537043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403206-8ee5-4fd5-83cf-61b41d80ec73 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:09.537464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403206-8ee5-4fd5-83cf-61b41d80ec73 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:09.537622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403206-8ee5-4fd5-83cf-61b41d80ec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:09.605187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:09.607860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-09403206-8ee5-4fd5-83cf-61b41d80ec73 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:09.609207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-09403206-8ee5-4fd5-83cf-61b41d80ec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:09.707551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:09.745715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:09.747318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:09.852354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326366c-ba2f-4d6b-abcc-cdbc2802f6ac None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:09.900718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:09.924885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:09.925505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:09.925786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:09.926063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:09.927727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:10.040352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:10.041810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f26e-e8f2-411d-a9ee-766822afed1c service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:10.059962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 727/1455] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:12.103830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635134a3-10b9-41a1-95db-feb84158048a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:12.104414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635134a3-10b9-41a1-95db-feb84158048a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:12.104414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635134a3-10b9-41a1-95db-feb84158048a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:12.105811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:12.110954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647c4e-1bbf-45f7-b345-e2875f524818 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:12.111400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647c4e-1bbf-45f7-b345-e2875f524818 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:12.111503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04647c4e-1bbf-45f7-b345-e2875f524818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:12.176797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-04647c4e-1bbf-45f7-b345-e2875f524818 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:12.178011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-04647c4e-1bbf-45f7-b345-e2875f524818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:12.206945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:12.273149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:12.312172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635134a3-10b9-41a1-95db-feb84158048a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:12.313748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-635134a3-10b9-41a1-95db-feb84158048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:12.418441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635134a3-10b9-41a1-95db-feb84158048a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:12.445449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:12.490324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:12.490944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:12.491223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:12.491495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:12.493707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:12.602163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:12.603603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbb2b-11e0-47ce-81fa-8fe6bd6062c0 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:12.621719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 730/1458] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:14.760276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:14.760835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:14.760835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:14.762131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:14.767458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21102e-b9f4-411d-b780-68f33788b861 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:14.767869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21102e-b9f4-411d-b780-68f33788b861 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:14.768014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21102e-b9f4-411d-b780-68f33788b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:14.832338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bb21102e-b9f4-411d-b780-68f33788b861 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:14.833576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bb21102e-b9f4-411d-b780-68f33788b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:14.841775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:14.930098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:14.968631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:14.970162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:15.075641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49a9b-9497-479e-a6eb-d46bc9f3b6ba None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:15.129836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:15.148218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:15.148867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:15.149167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:15.149437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:15.151055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:15.256205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:15.257650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ed76-45b6-42f9-b00d-11ba340b3ff5 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:15.276818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 730/1461] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:17.331527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:17.332008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:17.332123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:17.333507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:17.338868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b93df-4b34-4032-9e60-52397cdb52d6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:17.339309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b93df-4b34-4032-9e60-52397cdb52d6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:17.339413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b93df-4b34-4032-9e60-52397cdb52d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:17.404122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-512b93df-4b34-4032-9e60-52397cdb52d6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:17.405359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-512b93df-4b34-4032-9e60-52397cdb52d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:17.447077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:17.507421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:17.545182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:17.546693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:17.771495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45d651-f4e5-4998-8d26-fb8e6a4a0c1e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:17.821263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:17.859000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:17.859612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:17.859891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:17.860163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:17.861802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:17.972101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:17.973574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5195d-1f89-425d-820f-248ee322d40a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:17.991407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 733/1464] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:20.010929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:20.011459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:20.011459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:20.012887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:20.018155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661aa3b-32e6-4439-bba3-b6de49e597d9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:20.018583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661aa3b-32e6-4439-bba3-b6de49e597d9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:20.018724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661aa3b-32e6-4439-bba3-b6de49e597d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:20.082534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d661aa3b-32e6-4439-bba3-b6de49e597d9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:20.083752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d661aa3b-32e6-4439-bba3-b6de49e597d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:20.101029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:20.177193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:20.214250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:20.215823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:20.319063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef88de-298c-4fe2-9a39-7176a8da0540 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:20.383923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:20.390644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:20.391250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:20.391526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:20.391816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:20.393481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:20.502018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:20.503436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaf50f-f636-4ef0-8ab4-ec110f763f91 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:20.521307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 733/1467] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:22.533784 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:22.534304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:22.534415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:22.535660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:26:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:22.541387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79eb83-2743-4e6c-8911-c804b0d4b4b7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:22.541796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79eb83-2743-4e6c-8911-c804b0d4b4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:22.541946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79eb83-2743-4e6c-8911-c804b0d4b4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:22.608108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fe79eb83-2743-4e6c-8911-c804b0d4b4b7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:22.609369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fe79eb83-2743-4e6c-8911-c804b0d4b4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:22.659495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:22.707726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:22.745676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:22.747260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:22.860972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9e9a4-8e5d-4305-bc8a-527064e1d04b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:22.907909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:22.933909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:22.934522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:22.934799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:22.935098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:22.936746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:23.043008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:23.044493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bdfd3-e42b-4349-bf51-4a0ff7da0f9a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:23.062969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 736/1470] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:32.705894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=22 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:33.060737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=66 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:42.526333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a0a8-09be-4cde-957d-b2945a94c724 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:42.526877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a0a8-09be-4cde-957d-b2945a94c724 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:42.526877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a0a8-09be-4cde-957d-b2945a94c724 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:42.528233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 735/1471] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:42.533712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeac28-ff73-4b1e-bff0-2ffa4c462225 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:42.534547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeac28-ff73-4b1e-bff0-2ffa4c462225 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:42.534547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeac28-ff73-4b1e-bff0-2ffa4c462225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:42.600281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c0eeac28-ff73-4b1e-bff0-2ffa4c462225 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:42.601624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c0eeac28-ff73-4b1e-bff0-2ffa4c462225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:42.696008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 737/1472] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:42.778433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:42.778723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:42.779002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:42.780349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 736/1473] 10.222.0.144 () {54 vars in 971 bytes} [Tue Jul 26 16:26:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:42.785305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87275a-8b31-4be8-99a8-056d948492a9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:42.785594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87275a-8b31-4be8-99a8-056d948492a9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:42.785870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87275a-8b31-4be8-99a8-056d948492a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:42.860523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7c87275a-8b31-4be8-99a8-056d948492a9 None None] MFA Rules not processed for user `b6d652014d6446a0ae162120aac0ed94`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:42.861736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7c87275a-8b31-4be8-99a8-056d948492a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:42.886053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:42.941083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 738/1474] 10.222.0.144 () {58 vars in 1068 bytes} [Tue Jul 26 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:42.949367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:42.950911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:43.068475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1bc62-6e3c-42c0-8fd0-f645ee3eb2b6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:43.140438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:43.141053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:43.141333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:43.141609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:43.143235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:43.195139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:43.249035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:43.250462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95282f-907d-4b66-9136-8e87689d0da6 service glance] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:43.269183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 737/1475] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:45.337324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:45.337774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:45.337877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:45.339226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 739/1476] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:45.344303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28227e-544e-4b10-ac77-75e5ada0251b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:45.344784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28227e-544e-4b10-ac77-75e5ada0251b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:45.344891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28227e-544e-4b10-ac77-75e5ada0251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:45.413947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dd28227e-544e-4b10-ac77-75e5ada0251b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:45.415701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dd28227e-544e-4b10-ac77-75e5ada0251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:45.512272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 738/1477] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:45.534787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:45.536354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:45.658753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923d9af-6ebd-4333-a522-5b9198a1c320 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:45.715896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:45.762346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:45.762949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:45.763223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:45.763491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:45.764173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:45.765931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:26:45.765931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:26:45.765931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:26:45.767056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216857c-7227-40e1-aa1b-748bc6dca96a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:45.787469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 740/1478] 10.222.0.144 () {56 vars in 1113 bytes} [Tue Jul 26 16:26:45 2022] GET /identity/v3/projects => generated 2562 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:26:47.803634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:47.804251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:47.804251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:47.806135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 739/1479] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:47.811174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc085a-b60a-4681-a50e-5b4026681e2a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:47.811595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc085a-b60a-4681-a50e-5b4026681e2a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:47.811756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc085a-b60a-4681-a50e-5b4026681e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:47.875553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e6cc085a-b60a-4681-a50e-5b4026681e2a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:47.876810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6cc085a-b60a-4681-a50e-5b4026681e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:47.972533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 741/1480] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:48.015334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:48.016861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:48.026987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:48.159871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63938000-a69e-4a39-9c48-fc1fb45c8078 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:48.236559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:48.272338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:48.273431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:48.273938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:48.274428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:48.277577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:48.369398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:48.370981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7445cb-7e4a-4369-b340-6d558de5867c service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:48.389644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 740/1481] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:50.562386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:50.562938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:50.562938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:50.564263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 742/1482] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:50.569792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187165e8-afd5-4110-9bfd-820e476c1f3a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:50.570229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187165e8-afd5-4110-9bfd-820e476c1f3a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:50.570335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187165e8-afd5-4110-9bfd-820e476c1f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:50.635683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-187165e8-afd5-4110-9bfd-820e476c1f3a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:50.636919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-187165e8-afd5-4110-9bfd-820e476c1f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:50.683510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:50.733750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 741/1483] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:50.769781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:50.771318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:50.794639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:50.871959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31659f41-3da1-487e-a97f-7ddae4c9b33d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:50.941806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:50.942411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:50.942686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:50.942951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:50.944587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:51.018377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:51.052401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:51.053846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c506eb-fe89-4dee-a7c4-b7e94af36500 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:51.072656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 743/1484] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:53.139554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:53.140118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:53.140118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:53.141499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 742/1485] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:53.146692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df3004-cae9-4d55-8a4d-8c9bd7e2ea60 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:53.147142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df3004-cae9-4d55-8a4d-8c9bd7e2ea60 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:53.147250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df3004-cae9-4d55-8a4d-8c9bd7e2ea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:53.214300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a1df3004-cae9-4d55-8a4d-8c9bd7e2ea60 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:53.215525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a1df3004-cae9-4d55-8a4d-8c9bd7e2ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:53.312903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 744/1486] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:53.349944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:53.352024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:53.454241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6cef2-334a-4b35-9db9-2d78bccfb21f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:53.536576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:53.538362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:53.539530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:53.539809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:53.540078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:53.541760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:53.647453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:53.649406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cdda3-f45c-45f4-a794-989cc4d46ec1 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:53.667802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 743/1487] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:55.880156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:55.880707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:55.880835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:55.882179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 745/1488] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:55.888243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04817d-8c23-485c-95b7-4f983146ab0e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:55.888708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04817d-8c23-485c-95b7-4f983146ab0e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:55.888837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04817d-8c23-485c-95b7-4f983146ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:55.954239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8a04817d-8c23-485c-95b7-4f983146ab0e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:55.955453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8a04817d-8c23-485c-95b7-4f983146ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:56.011225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:56.054814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 744/1489] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:56.095101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:56.096677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:56.133720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:56.200984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e155-a9b2-4162-b991-011b2bd6e0e0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:56.275296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:56.275915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:56.276202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:56.276505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:56.278132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:56.359673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:56.392102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:56.393478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df4bb1-a181-480d-8db0-e5754a39e367 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:56.412259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 746/1490] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:26:58.820472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:58.821008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:58.821008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:58.822286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 745/1491] 10.222.0.144 () {54 vars in 931 bytes} [Tue Jul 26 16:26:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:26:58.827514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f4d63-6c00-46b0-8569-35eefdc9e3b3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:58.827907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f4d63-6c00-46b0-8569-35eefdc9e3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:58.828065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f4d63-6c00-46b0-8569-35eefdc9e3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:58.892215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-937f4d63-6c00-46b0-8569-35eefdc9e3b3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:26:58.893434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-937f4d63-6c00-46b0-8569-35eefdc9e3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:58.985997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 747/1492] 10.222.0.144 () {58 vars in 1028 bytes} [Tue Jul 26 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:26:59.025078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:26:59.026621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:59.133747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d5af1-321c-4e0e-ae62-5abe4844e476 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:26:59.205640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:26:59.206335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:26:59.206615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:26:59.206883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:26:59.208576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:26:59.218306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:26:59.316260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:26:59.317946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e0ab0-4986-4739-8321-00c2e4709f78 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:26:59.337527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 746/1493] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:27:08.984198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=82 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:27:09.334556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=50 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:41.303921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:41.304545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:41.304545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:41.380320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:28:41.381572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:41.479359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 748/1494] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:28:41.487314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:41.487823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:41.489332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:41.565684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:28:41.566863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:41.672402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 747/1495] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:28:41.677096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:41.678313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:41.689982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dead6e-3101-4d7b-93f6-95175fe2e551 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:41.750225 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:41.783675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:41.784298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:41.784605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:41.784881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:41.785566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:28:41.785870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:41.786874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:41.786874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:28:41.786874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:28:41.787763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:41.799054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 749/1496] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:28:41.808078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:41.817302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10adbb3-846e-455b-89cf-426fa06d01d7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:41.876761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:41.910159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:41.910774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:41.911349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:41.911790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:41.912792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:41.915033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:41.915033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:28:41.915033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:28:41.916208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:42.067422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 748/1497] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:28:41 2022] POST /identity/v3/projects => generated 380 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:28:42.071860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:42.073560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:42.107161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b94e23-4001-469b-bcfa-a9487026c104 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:42.199739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:42.200351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:42.200666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:42.200938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:42.201604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:42.202608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:42.202608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:28:42.202608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:28:42.203807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:42.298821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 750/1498] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:28:42 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:28:42.303020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:42.305063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:42.365637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fd5c2-d2e9-479a-a8ce-e5fce5efd3f8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:42.448530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:42.458656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:42.460252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:42.460252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:42.460252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:42.460709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:42.461529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:42.461529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:28:42.461529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:28:42.462397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:42.474645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 749/1499] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:28:42 2022] GET /identity/v3/roles => generated 1427 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:28:42.478918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:42.480514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:42.492610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a52de-7583-430d-8548-915bd2a331ea admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:42.518406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:42.587308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:42.587921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:42.588201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:42.588505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:42.589196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:42.590206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:42.590206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:28:42.590206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:28:42.591078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd706527-0210-4a89-b574-157b5578fa4b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:42.603575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 751/1500] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:28:42 2022] GET /identity/v3/roles => generated 1427 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:28:42.607838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:42.609425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:42.643023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf9743-222e-447c-b35e-90a76f58adfe admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:42.742516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:42.743171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:42.743445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:42.743710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] PATH_INFO: `/v3/projects/2c1550e7ee4740ea830dbb1e8c698f4a/users/5640d1a2b82c448ab8d1e2259e267890/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:42.764939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1550e7ee4740ea830dbb1e8c698f4a, user_id=5640d1a2b82c448ab8d1e2259e267890, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:42.765942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:42.765942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:28:42.765942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:28:42.767849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6d2eb-5327-4986-bbdd-15abc13b91c0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:42.864165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 750/1501] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:28:42 2022] PUT /identity/v3/projects/2c1550e7ee4740ea830dbb1e8c698f4a/users/5640d1a2b82c448ab8d1e2259e267890/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:28:42.877950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d3cb-53f8-4b45-843e-1d4e642f3b26 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:42.878240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d3cb-53f8-4b45-843e-1d4e642f3b26 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:42.878520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d3cb-53f8-4b45-843e-1d4e642f3b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:42.934936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2bb2d3cb-53f8-4b45-843e-1d4e642f3b26 None None] MFA Rules not processed for user `5640d1a2b82c448ab8d1e2259e267890`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:28:42.936066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2bb2d3cb-53f8-4b45-843e-1d4e642f3b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:43.011523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 752/1502] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:28:43.025304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:43.025593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:43.025867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:43.027214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 751/1503] 10.222.0.144 () {54 vars in 969 bytes} [Tue Jul 26 16:28:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:28:43.031984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5b93-c6a4-496c-ad14-eb3a38ce3731 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:43.032300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5b93-c6a4-496c-ad14-eb3a38ce3731 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:43.032612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5b93-c6a4-496c-ad14-eb3a38ce3731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:43.063153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:43.106839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1fad5b93-c6a4-496c-ad14-eb3a38ce3731 None None] MFA Rules not processed for user `5361a7650f164ff8b11abfe6b3c4c382`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:28:43.108046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1fad5b93-c6a4-496c-ad14-eb3a38ce3731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:43.181805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 753/1504] 10.222.0.144 () {58 vars in 1066 bytes} [Tue Jul 26 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:28:43.189740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:43.191207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:43.242166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:43.295009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60970fd3-b7fa-46b0-bb57-e671045f0f8c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:43.367553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:43.368157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:43.368530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:43.368807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:43.370416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:43.470199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:43.472372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:43.477135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:43.495593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 752/1505] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:28:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:28:43.545843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3386f-13db-443e-b7bb-2f8f0054b243 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:43.546270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3386f-13db-443e-b7bb-2f8f0054b243 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:43.546405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3386f-13db-443e-b7bb-2f8f0054b243 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:43.547661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 754/1506] 10.222.0.144 () {52 vars in 835 bytes} [Tue Jul 26 16:28:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:28:43.551449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:43.552671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:43.654976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26983-c16c-4a0b-8049-bfe057736957 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:43.751962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:43.752641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:43.752928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:43.753203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] PATH_INFO: `/v3/users/5640d1a2b82c448ab8d1e2259e267890` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:43.765763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5640d1a2b82c448ab8d1e2259e267890)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:43.766815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:43.766815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:28:43.766815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:28:43.768020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:43.778034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:43.970057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5640d1a2b82c448ab8d1e2259e267890'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:43.970586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5640d1a2b82c448ab8d1e2259e267890'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:44.037473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5640d1a2b82c448ab8d1e2259e267890'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:44.037846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5640d1a2b82c448ab8d1e2259e267890'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:44.050662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5640d1a2b82c448ab8d1e2259e267890'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:44.073888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 753/1507] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:28:43 2022] DELETE /identity/v3/users/5640d1a2b82c448ab8d1e2259e267890 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:28:44.078441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:44.078725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:44.078998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:44.133483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:28:44.134941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:44.208794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:44.221039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 755/1508] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:28:44.230755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:44.232011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:44.343065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b36763-b371-4434-bbc7-8213a7e87baa admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:44.395799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:44.415617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:44.416230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:44.416598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:44.416841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:44.418445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:44.528619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:44.530076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc2521-06ac-42dc-a928-964df58d8145 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:44.548225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 754/1509] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:28:45.100728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:28:45.104197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:28:45.113737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe938-3aff-481d-b85e-c3c91cf940db None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:28:45.211838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:28:45.212602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:28:45.213052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:28:45.213524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] PATH_INFO: `/v3/projects/2c1550e7ee4740ea830dbb1e8c698f4a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:28:45.225580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1550e7ee4740ea830dbb1e8c698f4a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:28:45.226624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:28:45.226624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:28:45.226624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:28:45.227830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:28:45.348073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1550e7ee4740ea830dbb1e8c698f4a', 'request_id': 'req-f78cc91b-3513-4b61-9180-f63bd70a0275'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:45.361014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1550e7ee4740ea830dbb1e8c698f4a', 'request_id': 'req-f78cc91b-3513-4b61-9180-f63bd70a0275'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:45.434610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1550e7ee4740ea830dbb1e8c698f4a', 'request_id': 'req-f78cc91b-3513-4b61-9180-f63bd70a0275'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:45.457888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] The token cache is being invalidate because project 2c1550e7ee4740ea830dbb1e8c698f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:28:45.458170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:45.458590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f78cc91b-3513-4b61-9180-f63bd70a0275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:28:45.460206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 756/1510] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:28:45 2022] DELETE /identity/v3/projects/2c1550e7ee4740ea830dbb1e8c698f4a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:28:54.545983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=62 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:28:55.455785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:30:28.909599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:28.910199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:28.910199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:28.911414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:30:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:28.916891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919ed9-647b-4e22-a89b-88b5d1e394dc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:28.917373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919ed9-647b-4e22-a89b-88b5d1e394dc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:28.917488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919ed9-647b-4e22-a89b-88b5d1e394dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:28.999717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f4919ed9-647b-4e22-a89b-88b5d1e394dc None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:29.001071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4919ed9-647b-4e22-a89b-88b5d1e394dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:29.108350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:29.261371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:29.264299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:29.389359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee8933-4888-4688-be1c-9f9cac81dcb5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:29.394999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:29.395669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:29.395956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:29.396256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:29.405601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:29.407201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39c220-9d5d-4929-99aa-3497681036ee service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:29.417637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 756/1513] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:30.254646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dba65-9797-4b82-876a-0640b0edcd28 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:30.255151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dba65-9797-4b82-876a-0640b0edcd28 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:30.255151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dba65-9797-4b82-876a-0640b0edcd28 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:30.256616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:30:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:30.261714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843299f-794d-49bc-88a4-b3284f1e12c5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:30.262120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843299f-794d-49bc-88a4-b3284f1e12c5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:30.262277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843299f-794d-49bc-88a4-b3284f1e12c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:30.282509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6843299f-794d-49bc-88a4-b3284f1e12c5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:30.283692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6843299f-794d-49bc-88a4-b3284f1e12c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:30.292014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:30.445258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dba65-9797-4b82-876a-0640b0edcd28 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:30.460738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dba65-9797-4b82-876a-0640b0edcd28 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:30.466912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:30.467590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:30.467941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:30.468308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:30.477306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:30.478896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e148563-9749-4728-915b-04e8590a02a9 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:30.489262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 759/1516] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:31.110200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f694-99b5-436e-bf73-e046258b3ad6 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:31.110490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f694-99b5-436e-bf73-e046258b3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:31.110834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900f694-99b5-436e-bf73-e046258b3ad6 None None] PATH_INFO: `/v3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:31.112339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 758/1517] 10.222.0.86 () {54 vars in 929 bytes} [Tue Jul 26 16:30:31 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:30:31.118771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6b47-a6f4-4a77-a692-359ea7165589 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:31.119061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6b47-a6f4-4a77-a692-359ea7165589 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:31.119370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6b47-a6f4-4a77-a692-359ea7165589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:31.183349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-119b6b47-a6f4-4a77-a692-359ea7165589 None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:31.184619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-119b6b47-a6f4-4a77-a692-359ea7165589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:31.202418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:30:31.260420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 760/1518] 10.222.0.86 () {58 vars in 1019 bytes} [Tue Jul 26 16:30:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:40.277003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=56 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:30:40.664546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06fc67-1538-4d2e-91e6-485f298ca69d None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:40.664834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06fc67-1538-4d2e-91e6-485f298ca69d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:40.665112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06fc67-1538-4d2e-91e6-485f298ca69d None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:40.666467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 759/1519] 10.222.0.144 () {54 vars in 966 bytes} [Tue Jul 26 16:30:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:40.671472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dcf04-fc49-4b25-91be-650ac2f43a0c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:40.671933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dcf04-fc49-4b25-91be-650ac2f43a0c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:40.672029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dcf04-fc49-4b25-91be-650ac2f43a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:40.691602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-882dcf04-fc49-4b25-91be-650ac2f43a0c None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:40.692849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-882dcf04-fc49-4b25-91be-650ac2f43a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:40.701339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 761/1520] 10.222.0.144 () {58 vars in 1063 bytes} [Tue Jul 26 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:40.709883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06fc67-1538-4d2e-91e6-485f298ca69d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:40.723319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06fc67-1538-4d2e-91e6-485f298ca69d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:40.728247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:40.728924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:40.729203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:40.729474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:40.737683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:40.739239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820abfd6-5b37-42b6-a0b0-d34b316a6c8e service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:40.749033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 760/1521] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:41.709229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241bd-4a5c-458c-a577-738ea1206f8e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:41.709706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241bd-4a5c-458c-a577-738ea1206f8e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:41.709838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241bd-4a5c-458c-a577-738ea1206f8e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:41.711176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:30:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:30:41.716820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863b46-e478-4289-a9a4-585ab79a5e2e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:41.717666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863b46-e478-4289-a9a4-585ab79a5e2e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:41.718130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863b46-e478-4289-a9a4-585ab79a5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:41.738819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d1863b46-e478-4289-a9a4-585ab79a5e2e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:41.740097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d1863b46-e478-4289-a9a4-585ab79a5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:41.748513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:41.902396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c241bd-4a5c-458c-a577-738ea1206f8e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:41.917298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c241bd-4a5c-458c-a577-738ea1206f8e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:41.923350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:41.923955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:41.924233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:41.924539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:41.932589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:41.937880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1037-a295-4167-b618-dc03447d19d5 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:41.943481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 763/1524] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:42.765797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a76b7-df47-433c-8a95-fee65013a252 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:42.766329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a76b7-df47-433c-8a95-fee65013a252 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:42.766329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a76b7-df47-433c-8a95-fee65013a252 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:42.767657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 16:30:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:42.772482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870f3f9-95c2-4394-a4ec-e4561bdd46ec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:42.772864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870f3f9-95c2-4394-a4ec-e4561bdd46ec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:42.773046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870f3f9-95c2-4394-a4ec-e4561bdd46ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:42.791835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9870f3f9-95c2-4394-a4ec-e4561bdd46ec None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:42.793293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9870f3f9-95c2-4394-a4ec-e4561bdd46ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:42.801182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:42.953149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a76b7-df47-433c-8a95-fee65013a252 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:42.967391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a76b7-df47-433c-8a95-fee65013a252 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:42.972843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:42.973443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:42.973726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:42.974002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:42.982047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:42.983623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05283700-d0f3-459a-a77d-3a879f4e3c85 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:42.993263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 763/1527] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:46.945800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:46.946399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:46.946399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:46.947708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:30:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:46.952941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab259e-8027-47d0-80c8-f60f47b36716 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:46.953354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab259e-8027-47d0-80c8-f60f47b36716 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:46.953498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab259e-8027-47d0-80c8-f60f47b36716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:46.973539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9eab259e-8027-47d0-80c8-f60f47b36716 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:46.974755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9eab259e-8027-47d0-80c8-f60f47b36716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:46.982744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:47.057641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:47.059401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:47.152119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86670872-2c07-4195-a9b1-bf8c67220e98 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:47.156337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:47.156979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:47.157274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:47.157548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:47.166517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:47.167951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6515d-794a-48df-91ba-8d2756b7f859 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:47.177407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 766/1530] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:49.176728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120ea6-5715-4fd1-bc0b-2b5e660432e2 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:49.179556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120ea6-5715-4fd1-bc0b-2b5e660432e2 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:49.179556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120ea6-5715-4fd1-bc0b-2b5e660432e2 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:49.179556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:30:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:49.183822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f338a60-0765-4dba-ab5e-5c9641e179e6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:49.184279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f338a60-0765-4dba-ab5e-5c9641e179e6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:49.184418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f338a60-0765-4dba-ab5e-5c9641e179e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:49.203830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1f338a60-0765-4dba-ab5e-5c9641e179e6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:49.205056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1f338a60-0765-4dba-ab5e-5c9641e179e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:49.212807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:49.249119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120ea6-5715-4fd1-bc0b-2b5e660432e2 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:49.260268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120ea6-5715-4fd1-bc0b-2b5e660432e2 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:49.264120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:49.265245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:49.265537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:49.265809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:49.274757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:49.276210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42f404-85b6-4e5a-bebc-295fbfd542c8 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:49.286215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 766/1533] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:52.817747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:52.818281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:52.818281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:52.819553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:30:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:52.824680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a27f8-20ee-4a7f-8629-eb1eb8155a23 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:52.825135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a27f8-20ee-4a7f-8629-eb1eb8155a23 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:52.825276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a27f8-20ee-4a7f-8629-eb1eb8155a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:52.845670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-621a27f8-20ee-4a7f-8629-eb1eb8155a23 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:52.846857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-621a27f8-20ee-4a7f-8629-eb1eb8155a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:52.854639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:53.167310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:53.169034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:53.235866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:30:53.249418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e986-5fec-42a7-9172-fb0fe5c00244 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:53.253514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:53.254134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:53.254418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:53.254694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:53.263800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:53.265869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846186a0-997a-4c6c-afa0-3ec2fb0cb88d service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:53.275139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 769/1536] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:30:55.304471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:55.304979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:55.304979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:55.306314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 16:30:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:30:55.311404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f5b0-f400-467c-9945-36f14bf2a029 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:55.311853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f5b0-f400-467c-9945-36f14bf2a029 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:55.311996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f5b0-f400-467c-9945-36f14bf2a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:55.331792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8578f5b0-f400-467c-9945-36f14bf2a029 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:30:55.333017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8578f5b0-f400-467c-9945-36f14bf2a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:55.340662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:30:55.654127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:30:55.655814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:30:55.668688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f169c3-f6f9-4b38-bdbd-9cb832ed7f13 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:30:55.672495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:30:55.673212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:30:55.673498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:30:55.673774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:30:55.682878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:30:55.684315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39e361-c404-4d08-aeac-a46dfddcf6e7 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:30:55.694806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 769/1539] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:31:05.327135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=11 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:31:05.690296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=33 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:27.389264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.389930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.389930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.427365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.428610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.440641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 771/1540] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:27.449687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.450313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.450313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.485417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.486651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.507142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 770/1541] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:27.512056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:27.526244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfe8ad-f231-42f5-be02-4fd616e7d314 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:27.531190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:27.532036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.532544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.533034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.534034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:27.534560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:27.536346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da029c1-2340-4a69-811e-c229d57e0e61 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:27.556994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 772/1542] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:27.565893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:27.579134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62809-7863-4b53-8fde-68ff33d1b30a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:27.589561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:27.590195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.590475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.590747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.591439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:27.593487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:27.678449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec05ac-753b-4e42-8400-5683e142e464 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.679096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec05ac-753b-4e42-8400-5683e142e464 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.679587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec05ac-753b-4e42-8400-5683e142e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.708191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-16ec05ac-753b-4e42-8400-5683e142e464 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.709563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-16ec05ac-753b-4e42-8400-5683e142e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.711683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 771/1543] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/projects => generated 378 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:27.716766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:27.818293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 773/1544] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:27.819736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b100359-1627-4e1f-bf6b-159917d099f6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.820029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b100359-1627-4e1f-bf6b-159917d099f6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.820312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b100359-1627-4e1f-bf6b-159917d099f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.835729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f2ac-8e09-4e24-bda4-4db365863c2a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:27.850502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:27.851139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.851421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.851924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.852630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:27.853685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:32:27.853685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:32:27.853685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:32:27.854975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4334de0-48dd-46e1-8ade-b1344cebe548 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:27.860146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3b100359-1627-4e1f-bf6b-159917d099f6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.861414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b100359-1627-4e1f-bf6b-159917d099f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.882231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 774/1545] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:27.883511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c349-fb42-4ba6-a8d3-59505600f2a8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.883801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c349-fb42-4ba6-a8d3-59505600f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.884079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c349-fb42-4ba6-a8d3-59505600f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.915152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-03b1c349-fb42-4ba6-a8d3-59505600f2a8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.916406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-03b1c349-fb42-4ba6-a8d3-59505600f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.925048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 775/1546] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:27.926431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35966b4-c474-4d0e-b15d-66a6772d8e03 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.926715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35966b4-c474-4d0e-b15d-66a6772d8e03 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.930935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35966b4-c474-4d0e-b15d-66a6772d8e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.958441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 772/1547] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:32:27.959846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f0947-f606-4ab9-82d6-09f7b74ca8b9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.960137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f0947-f606-4ab9-82d6-09f7b74ca8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.960423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f0947-f606-4ab9-82d6-09f7b74ca8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.962947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d35966b4-c474-4d0e-b15d-66a6772d8e03 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.964171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d35966b4-c474-4d0e-b15d-66a6772d8e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.981066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 776/1548] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:27.981066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4e52-3471-4ce4-8129-a719d4df0172 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.981066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4e52-3471-4ce4-8129-a719d4df0172 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.981066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4e52-3471-4ce4-8129-a719d4df0172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:27.982422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-816f0947-f606-4ab9-82d6-09f7b74ca8b9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:27.983633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-816f0947-f606-4ab9-82d6-09f7b74ca8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:27.995631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 773/1549] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:27.998006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8256-e141-4926-a1cc-5cd6c0c70524 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:27.998766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8256-e141-4926-a1cc-5cd6c0c70524 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:27.999318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8256-e141-4926-a1cc-5cd6c0c70524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.000206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a46a4e52-3471-4ce4-8129-a719d4df0172 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.001442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a46a4e52-3471-4ce4-8129-a719d4df0172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.010037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 777/1550] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.011227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a760ded-07dc-4622-b4bd-5a39edb93279 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.011511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a760ded-07dc-4622-b4bd-5a39edb93279 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.011783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a760ded-07dc-4622-b4bd-5a39edb93279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.023560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-824e8256-e141-4926-a1cc-5cd6c0c70524 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.024765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-824e8256-e141-4926-a1cc-5cd6c0c70524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.030829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2a760ded-07dc-4622-b4bd-5a39edb93279 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.031993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2a760ded-07dc-4622-b4bd-5a39edb93279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.032602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 774/1551] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.033773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.034055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.034328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.042301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 778/1552] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.043635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.043920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.044193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.058198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.059514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.063591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.064771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.068024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 775/1553] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.068826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.076076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 779/1554] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.076076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.080786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f347d-4605-4b44-9aee-5f511767f500 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.085709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.086323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.086607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.086884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.087258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227c93d-3851-4459-a4af-8d4cb54b98b0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.087648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.089345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240977e-5d50-4fe8-9a1c-e6ec10ab9317 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.092722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.092722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.092992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.093272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.093961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.094265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.095844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58b0a-d4ec-4406-9ad6-b52ca102e682 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.100126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 776/1555] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:28.101372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.101663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.101941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.109291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 780/1556] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.110419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae2959-0a44-44e4-93cd-eef2e1429430 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.110768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae2959-0a44-44e4-93cd-eef2e1429430 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.111704 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae2959-0a44-44e4-93cd-eef2e1429430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.122400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.127213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.129581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cdae2959-0a44-44e4-93cd-eef2e1429430 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.130761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cdae2959-0a44-44e4-93cd-eef2e1429430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.131501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 777/1557] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.132443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.139894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 781/1558] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.141173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295f67-7652-496f-9e02-e0a57c426eaa None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.141459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295f67-7652-496f-9e02-e0a57c426eaa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.141730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295f67-7652-496f-9e02-e0a57c426eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.142591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba949f-e25f-4e29-8c80-c87d40af2d45 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.148219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.148850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.149126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.149396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.150085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.150408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.151410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:32:28.151410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:32:28.151410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:32:28.152319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.161332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bb295f67-7652-496f-9e02-e0a57c426eaa None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.162583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bb295f67-7652-496f-9e02-e0a57c426eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.164426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 778/1559] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.165236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.174972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 782/1560] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.175650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6165-485e-483e-9305-906885e3699d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.176838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.177125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.177425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.181018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.181550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.181827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.182099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.182777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.183080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.184681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.196579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:28.196904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 779/1561] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.197758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.197883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:28.206335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 783/1562] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:28.206944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.207314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496362ec-602f-4006-b1f2-56d56c93cf7d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.212254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.212906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.213187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.213460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.214166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.215729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.216223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95045b-0850-4d30-a656-5c9a85b6e26d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.220952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.221526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.221800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.222067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.222714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.223720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:32:28.223720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:32:28.223720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:32:28.224910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.226079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 780/1563] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:28.229352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.239766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c39f8-078e-4408-a9b7-37e53e0173ee admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.244429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.245038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.245322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.245599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.246286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.246594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.248221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.260817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 781/1564] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.261627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.273081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009c1d-4b1e-4fe0-8b50-793500ef7a17 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.277998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.278621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.278913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.279193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.279902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.280212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.281916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.298232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 782/1565] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.299043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.309722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd5b59-a89f-411f-88f7-118970fe3322 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.314584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.315193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.315480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.315809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.317030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.318918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.329021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:28.353544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 784/1566] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 374 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:28.354513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.401563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:28.416187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b3e4-3054-4294-b134-b2dd5afbc899 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.420942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.421542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.421820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.422093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.422773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.423074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.424734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.436688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 783/1567] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 386 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:28.436688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.437379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 785/1568] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.438199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.448374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee56b-427d-4a63-952d-61409aca2480 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.501480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dd221-04af-4359-8441-afc4c0956cbb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.506058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.506217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.506692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.506828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.506957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.507108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.507219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.507314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.507900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.508010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:28.508283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.509824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.509946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.521981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 786/1569] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:28.522822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.533800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45598-0fad-4ed8-a01a-ed542c3cb574 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.538942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.539582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.539860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.540133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] PATH_INFO: `/v3/projects/ce9690eefda448588e15513aae708a58/users/62c98d94542947a4842d4ad9c5123153/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.554906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9690eefda448588e15513aae708a58, user_id=62c98d94542947a4842d4ad9c5123153, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.555927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:32:28.555927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:32:28.555927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:32:28.557877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.633989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 784/1570] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 376 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:28.635001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.649380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 787/1571] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:28 2022] PUT /identity/v3/projects/ce9690eefda448588e15513aae708a58/users/62c98d94542947a4842d4ad9c5123153/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:28.650282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.704172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a307e7-aaef-4bb1-acc3-487fc85b96d9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.719689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412ef90-ac08-4535-8318-e9c8d846aea5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.724856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.725466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.725741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.726011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.726663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.728620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.761633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.767195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.767195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.767195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.767195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.767195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.845287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 788/1572] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 392 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:28.846281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.857460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 785/1573] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 386 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:28.858909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:28.874118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:28.887724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:28.911948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6058f05-b494-4434-9f9a-64751ad7e584 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.936029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b747-9ff9-4db8-9bd5-7814e1422e15 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:28.940878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.941579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.941857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.942129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.942787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.945036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:28.968481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:28.969191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:28.969643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:28.970373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:28.971033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:28.973682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.068049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 786/1574] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/projects => generated 378 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.068893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.106772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 789/1575] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:28 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.110979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.117107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc393d-e93f-4768-ba86-7016fa25f499 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.132336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1898d-416c-4ddf-bfe2-d36783b83c2a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.137243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.137885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.138190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.138722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.139882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.141838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.173506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.174143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.174426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.174702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.175521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.177453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320251d-dccd-44c9-ac6b-1a63a2801d39 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.261534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 787/1576] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.262409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.272856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3111d-c10c-4344-bb6c-0464b33cef57 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.277481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.278062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.278344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.278612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.279257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.281190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.302413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 790/1577] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/projects => generated 374 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.303747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.304043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.311893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.331007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] MFA Rules not processed for user `62c98d94542947a4842d4ad9c5123153`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:29.332833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:29.369488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 788/1578] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.370361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.376534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:29.381007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7409-6d8c-41d4-ad10-f50cade96199 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.397487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 791/1579] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:29.398293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.437358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.437972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.438254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.438526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.439180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.441647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.451229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:29.457867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecc530-8fd0-4c2c-9980-c8d0cf02c8d1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.462536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.467450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.467450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.467450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.467450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.467450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.565804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 789/1580] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.565804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.575315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aad5a-80cf-4ddf-994a-12619ac94328 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.575976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 792/1581] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.576633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.583082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.583692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.583972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.584554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.585275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.586200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bc1-7936-4941-8dd1-7d5bf520e050 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.587239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.590746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.591338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.591620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.591886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.592597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.594161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.604170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 793/1582] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.605062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.615451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6746ea-5892-4908-8790-0c49b052fa73 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.619994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.620579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.620857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.621126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.621776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.623299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.633924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 794/1583] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.634717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.644312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efb8f9-ef92-4acd-a558-28ae856e6fbe admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.648932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.649521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.649797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.650065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.650695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.652544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.696489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 790/1584] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.697319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.706964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcb46d-e91c-468f-9881-1f25e16f4329 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.711594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.712171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.712484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.712758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.713435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.715009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.724895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 791/1585] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.725640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.735350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec36b4-d136-4e21-bbc5-1e0182cbdec0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.735525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 795/1586] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.736286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.742975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.746619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.746619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.746619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.746619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.746938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02989d6-9eaf-453f-ae78-278ba0812e3c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.747033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.751376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.752199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.752523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.752803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.753545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.755677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.766155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 796/1587] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.770006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.779394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ac8cf-e4bd-4ef4-963d-89b370558a1a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.784090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.784832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.785115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.785407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.786081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.787646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.797884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 797/1588] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.802504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.808285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e813d-d0b3-4d20-a66b-80af8b66892a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.812888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.813448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.813781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.814058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.814712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.816329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.825801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 798/1589] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.826553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.836174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa0e7-027a-425f-8075-702fd5dcd902 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.840768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.841344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.841616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.841891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.842547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.844086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.848867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 792/1590] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/projects => generated 378 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:29.848867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.857026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 799/1591] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.857026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.867270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd9921-cf87-41c8-b334-a08dfec3d240 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.910043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b53d82-53ba-445b-b135-fc6ac8d55e35 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.914289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:29.915410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.916158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.916473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.916913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.917945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.919839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.923037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.923653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.923935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.924208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.924959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.926630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.930346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 793/1592] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.937367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.937481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 800/1593] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.937481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.945436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d03f-a564-4645-a40b-6a7517447fc9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.947422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1b606-c774-49a2-a606-cafb101c450c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.950465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.951085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.951479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.951662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.952510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.952843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.953849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.954144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.954248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.954513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.955212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.957476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.964883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 794/1594] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.965684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.968769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 801/1595] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.969774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:29.977408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5696-f9d8-4a7d-9fef-741811770a25 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.983219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9891c-81d9-4e30-bea0-1236328918cb admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:29.983370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.983370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.983370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.983370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.984061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:29.985691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:29.985828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:29.986402 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:29.986824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:29.987128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] PATH_INFO: `/v3/projects/14289132bbb4432f9afaa3376835eb8e/users/dfa6a4b037964f22b6f9ee25acf6a094/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:29.996689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 795/1596] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:29 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:29.999950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.002128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14289132bbb4432f9afaa3376835eb8e, user_id=dfa6a4b037964f22b6f9ee25acf6a094, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.005597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.008082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c231e-8620-411e-be58-df3ead292322 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.012913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.013853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.014068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.017598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.017598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.017598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.100116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 802/1597] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:29 2022] PUT /identity/v3/projects/14289132bbb4432f9afaa3376835eb8e/users/dfa6a4b037964f22b6f9ee25acf6a094/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.100868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.141637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 796/1598] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:29 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:30.142540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.154918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76896-52fe-4acc-bbda-1bc1dbb95656 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.160954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.161553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.161832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.162105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.162791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.164156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953edc5-2d99-4530-8245-91589dfb27d8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.164370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.169133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.170248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.170556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.170817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] PATH_INFO: `/v3/projects/23bea48eb25e4818bce5b81884348840/users/10dd7a84ac7b4cde9bef32976f374386/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.174661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 797/1599] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:30 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:30.176170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.185698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c94f9-5757-419e-b28c-fc94d29613c3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.187714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23bea48eb25e4818bce5b81884348840, user_id=10dd7a84ac7b4cde9bef32976f374386, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.190473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.190855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.191494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.191774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.192047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] PATH_INFO: `/v3/projects/8bf096b68d664989bbb0a08e0b5c4e4e/users/e4f115425e784615aa55c1c6a3a60aac/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.206464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf096b68d664989bbb0a08e0b5c4e4e, user_id=e4f115425e784615aa55c1c6a3a60aac, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.209124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.301287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 803/1600] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/23bea48eb25e4818bce5b81884348840/users/10dd7a84ac7b4cde9bef32976f374386/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.302151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.306883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 798/1601] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/8bf096b68d664989bbb0a08e0b5c4e4e/users/e4f115425e784615aa55c1c6a3a60aac/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.308260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.337447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:30.371338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1d236-b501-48f2-a19d-3ec093f95a43 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.375412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d0345-4350-4425-81e2-99e3f724a2a1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.375976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.376622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.376906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.377181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.377865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.379560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.380520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.382672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.382672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.382672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] PATH_INFO: `/v3/projects/08a954085f8f454698c55faa9351ad6e/users/07e70b1c968443eab469777283b8fb50/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.389735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 804/1602] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:30 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:30.390518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.397549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a954085f8f454698c55faa9351ad6e, user_id=07e70b1c968443eab469777283b8fb50, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.400256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cb7cb-2f84-4191-86a8-f6142e2c584a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.400927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835ee6b-cfe8-456e-bbab-343bda8b660c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.405549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.406190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.406471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.406743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] PATH_INFO: `/v3/projects/8bb1d794693b410d8c759796d5e20281/users/4c6fffcc3c0d4c49848fa6c2f6b90130/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.422037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb1d794693b410d8c759796d5e20281, user_id=4c6fffcc3c0d4c49848fa6c2f6b90130, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.424681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.500704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 799/1603] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/08a954085f8f454698c55faa9351ad6e/users/07e70b1c968443eab469777283b8fb50/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.503129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.503129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.503129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.529641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 805/1604] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/8bb1d794693b410d8c759796d5e20281/users/4c6fffcc3c0d4c49848fa6c2f6b90130/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.530467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.530726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] MFA Rules not processed for user `dfa6a4b037964f22b6f9ee25acf6a094`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.531945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.544405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:30.595090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfda1-4e0a-46ba-8487-a4a0fc7c49d8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.598549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 800/1605] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:30.599340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.605858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.606613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.606904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.607181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.607938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.609570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b88db-a81a-4183-bc16-8e0ecdc67da2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.614617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a82f0d-426b-40a4-a2bc-fc6dec66475a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.622723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 806/1606] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:30 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:30.623104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.623833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.624158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.624263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.624486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] PATH_INFO: `/v3/projects/940d1a5ec6514d18946628b99f16a96e/users/db18d0211f2a4e1c8949726ded7a9bb3/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.624589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.625201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.642775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=940d1a5ec6514d18946628b99f16a96e, user_id=db18d0211f2a4e1c8949726ded7a9bb3, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.646558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01b6fb-821f-4889-866e-b7c7a5dad31c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.657852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] MFA Rules not processed for user `10dd7a84ac7b4cde9bef32976f374386`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.659113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.707494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 801/1607] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/940d1a5ec6514d18946628b99f16a96e/users/db18d0211f2a4e1c8949726ded7a9bb3/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.709622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b54-9ef6-45ef-93c5-e0539edc37e1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.709821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b54-9ef6-45ef-93c5-e0539edc37e1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.710096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a6b54-9ef6-45ef-93c5-e0539edc37e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.728715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 807/1608] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:30.729746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.743936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ea77-9556-4ef7-a47c-385ecfe645ea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.746370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d18a6b54-9ef6-45ef-93c5-e0539edc37e1 None None] MFA Rules not processed for user `e4f115425e784615aa55c1c6a3a60aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.747615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d18a6b54-9ef6-45ef-93c5-e0539edc37e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.804511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.805180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.805463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.805738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] PATH_INFO: `/v3/projects/4088561009754a839604e3798787d8b9/users/1b9d76409f834b478981c0bbc0c3d4b6/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.817280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 802/1609] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:30.822236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4088561009754a839604e3798787d8b9, user_id=1b9d76409f834b478981c0bbc0c3d4b6, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.822972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524805-3b98-4029-8f09-216491624410 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.823267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524805-3b98-4029-8f09-216491624410 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.823545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524805-3b98-4029-8f09-216491624410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.825106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c9912-7c40-4bd0-8265-4f12e3a64444 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.830779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:30.843021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-70524805-3b98-4029-8f09-216491624410 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.844219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-70524805-3b98-4029-8f09-216491624410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.851836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 803/1610] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:30.853166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.853453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.853737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.872049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.873304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.881457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 804/1611] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:30.882228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:30.892100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fcf85-cfba-4ede-810d-10a2b8cbc607 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:30.899469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:30.900255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.900611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.900889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] PATH_INFO: `/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0/users/82b68d1ae63b4723a059868ff61a84b3/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.915585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4146f62417ba416b9f4d730bcc4e9ff0, user_id=82b68d1ae63b4723a059868ff61a84b3, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:30.918155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 808/1612] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/4088561009754a839604e3798787d8b9/users/1b9d76409f834b478981c0bbc0c3d4b6/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:30.918312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:30.923005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2c3d9-ad0d-47e6-b15e-2c70e929e6a7 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.926354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2c3d9-ad0d-47e6-b15e-2c70e929e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.926354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2c3d9-ad0d-47e6-b15e-2c70e929e6a7 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.926354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 809/1613] 10.222.0.144 () {54 vars in 967 bytes} [Tue Jul 26 16:32:30 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:30.926354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1810-0e01-48c3-880c-e407948dbc28 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:30.926354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1810-0e01-48c3-880c-e407948dbc28 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:30.926837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1810-0e01-48c3-880c-e407948dbc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:30.945445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1b1b1810-0e01-48c3-880c-e407948dbc28 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:30.946597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b1b1810-0e01-48c3-880c-e407948dbc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:30.958557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:31.004656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 810/1614] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:31.005885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.006172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.006451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.007915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 811/1615] 10.222.0.144 () {54 vars in 967 bytes} [Tue Jul 26 16:32:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:31.008465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.010047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.020859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 805/1616] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:30 2022] PUT /identity/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0/users/82b68d1ae63b4723a059868ff61a84b3/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:31.021750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.023269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.069081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e323d00-135e-4631-b59e-235d82f0feb5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.079067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee1644-29cf-469d-806a-c3969cb3640a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.082755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.083364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.083643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.083911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.119360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.119977 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.120258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.120574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.148490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.148490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c58025-296d-442d-8e80-67418baa34b1 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.156524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 806/1617] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.157942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defff31-4f0c-4172-b8a8-ca08b8121def None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.158233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defff31-4f0c-4172-b8a8-ca08b8121def None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.158518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defff31-4f0c-4172-b8a8-ca08b8121def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.179311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7defff31-4f0c-4172-b8a8-ca08b8121def None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:31.180561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7defff31-4f0c-4172-b8a8-ca08b8121def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.183524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.184998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13485ea-bbe5-43c1-8484-8069c8403c55 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.192043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 807/1618] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:31.192043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.192043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.192043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.200218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 812/1619] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.201550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.201895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.202105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.232410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:31.232642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:31.233630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.233865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.247982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:31.286694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:31.310185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 808/1620] 10.222.0.144 () {58 vars in 1064 bytes} [Tue Jul 26 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:31.311034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.319910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 813/1621] 10.222.0.144 () {58 vars in 1064 bytes} [Tue Jul 26 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:31.321436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218a7da-8449-4062-ad4a-2d2a9c7f2519 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.324552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.326196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.327000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.327383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.327597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.332090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4e208-4a20-4a52-b733-b7207119ad9c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.336981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.337668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.338020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.338365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.339236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.340316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.341036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.342030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.353611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 814/1622] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:31.355140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.357740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 809/1623] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.358674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.368748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660e91-ca96-485a-b43d-40b87a8c7288 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.371564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87dc58-616f-4f1d-831e-af72e794c699 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.372701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.373289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.373569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.373844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.377534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.378150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.380798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.380798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.383976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.385450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.395903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 815/1624] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.398781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.415285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9d4a4-a202-48c3-9d49-f25077b0f477 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.422392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.423991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.424187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.424473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.451541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.453877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.466679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 810/1625] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.468125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.487440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4455ef-7e12-462b-9bba-fc3e0e1a47af service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.496429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.497274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.497621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.498403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] PATH_INFO: `/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0/users/82b68d1ae63b4723a059868ff61a84b3/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.499379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.500895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4146f62417ba416b9f4d730bcc4e9ff0, user_id=82b68d1ae63b4723a059868ff61a84b3, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.502131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d031dc-1c0b-4e27-86f0-23b53b8ab4d3 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.515838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:31.517335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 816/1626] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:31.545366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 811/1627] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:31 2022] PUT /identity/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0/users/82b68d1ae63b4723a059868ff61a84b3/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:31.556609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f9a0b-1f5f-46da-a9e0-52752e89a196 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.557094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f9a0b-1f5f-46da-a9e0-52752e89a196 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.557316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f9a0b-1f5f-46da-a9e0-52752e89a196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.593028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-707f9a0b-1f5f-46da-a9e0-52752e89a196 None None] MFA Rules not processed for user `82b68d1ae63b4723a059868ff61a84b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:31.594418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-707f9a0b-1f5f-46da-a9e0-52752e89a196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.701623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 817/1628] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:31.713263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:31.872256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbdea-7a9e-4c24-a4fd-7d0c57601d34 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.872787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbdea-7a9e-4c24-a4fd-7d0c57601d34 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.873251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cbdea-7a9e-4c24-a4fd-7d0c57601d34 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.875439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 818/1629] 10.222.0.144 () {54 vars in 917 bytes} [Tue Jul 26 16:32:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:31.885657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.885657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.885657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.885884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef3741-6054-4c9d-98b6-3bac058bafb4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:31.890135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:31.890796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:31.891159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:31.891440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:31.933632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:31.935448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:31.941361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:31.996424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:31.997889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bad13-94de-442d-a2fb-df30e4b58360 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:32.007710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 812/1630] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:32.009105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209eaa1-7349-4c10-adaa-e58e9dea25e8 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:32.009400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209eaa1-7349-4c10-adaa-e58e9dea25e8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:32.009678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209eaa1-7349-4c10-adaa-e58e9dea25e8 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:32.010951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 813/1631] 10.222.0.144 () {54 vars in 917 bytes} [Tue Jul 26 16:32:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:32.018689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 819/1632] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:32.020940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:32.021512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:32.021985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:32.029564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:32.041950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ad6ef-e528-429f-b874-8cdbc7375bad None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:32.045795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:32.046514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:32.049793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:32.049793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:32.054131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:32.055345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:32.057976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:32.059399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:32.066304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 814/1633] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:32:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:32.087011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 820/1634] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:33.206280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:33.217342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c21f7-6991-4959-86ef-b4f47688be53 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:33.231570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:33.232460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:33.232775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:33.233196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:33.241357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:33.245077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:33.246594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:33.257371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 815/1635] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:33.475064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:33.487415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c4edc-f120-4b37-9132-f80e642b4999 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:33.494572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:33.495248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:33.495414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:33.495676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:33.615408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:33.625525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78246efe-f889-49cb-acdc-2e4d691b754a service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:33.640769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 821/1636] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:34.119937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:34.119937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f89e7-a591-4196-8e9c-44210fe3cd85 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:34.119937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:34.119937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:34.128066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:34.128066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] PATH_INFO: `/v3/users/e4f115425e784615aa55c1c6a3a60aac` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:34.128066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f115425e784615aa55c1c6a3a60aac)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:34.128066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:34.559644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f115425e784615aa55c1c6a3a60aac'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:34.560546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f115425e784615aa55c1c6a3a60aac'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:34.655173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f115425e784615aa55c1c6a3a60aac'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:34.655851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f115425e784615aa55c1c6a3a60aac'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:34.675074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f115425e784615aa55c1c6a3a60aac'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:34.709952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 816/1637] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:32:33 2022] DELETE /identity/v3/users/e4f115425e784615aa55c1c6a3a60aac => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:34.715360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:34.715909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:34.715909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:34.762692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:34.764515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:34.863856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 822/1638] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:34.911646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:34.913355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:35.049761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2da5c-3dc1-486b-8c42-79237687291f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:35.057276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:35.060042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:35.060042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:35.060042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:35.073899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:35.076139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca4ae9-78ab-49f7-b12d-bf6d48734494 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:35.112049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 817/1639] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:36.812004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:36.826110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:36.908697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5b147-15ac-4284-81f9-bb255b5ca499 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:36.932136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:36.933073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:36.936057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:36.940949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] PATH_INFO: `/v3/projects/8bf096b68d664989bbb0a08e0b5c4e4e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:36.941797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf096b68d664989bbb0a08e0b5c4e4e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:36.952970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:37.059059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:37.266243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf096b68d664989bbb0a08e0b5c4e4e', 'request_id': 'req-3ec82003-9453-498e-8d90-0be30e0fcb7b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:37.287095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf096b68d664989bbb0a08e0b5c4e4e', 'request_id': 'req-3ec82003-9453-498e-8d90-0be30e0fcb7b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:37.414931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf096b68d664989bbb0a08e0b5c4e4e', 'request_id': 'req-3ec82003-9453-498e-8d90-0be30e0fcb7b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:37.436088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] The token cache is being invalidate because project 8bf096b68d664989bbb0a08e0b5c4e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:32:37.436637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:37.438550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ec82003-9453-498e-8d90-0be30e0fcb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:37.447513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 823/1640] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:32:36 2022] DELETE /identity/v3/projects/8bf096b68d664989bbb0a08e0b5c4e4e => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:37.453274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:37.453838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:37.453838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:37.495458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:37.496588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:37.520902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:37.622452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:37.622832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:37.623190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:37.623724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 818/1641] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:37.669577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:37.671172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:37.703215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 824/1642] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:37.710042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:37.748252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67148d5-546e-48e3-bdcb-15319abead72 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:37.805638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:37.805638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:37.805638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:37.805894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:37.806754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:37.807064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:37.819520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:37.843910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 819/1643] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:37.844555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:37.861257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ee9a3-f6a0-4032-ad55-b2d21afae7c9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:37.867304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:37.868178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:37.868722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:37.869452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:37.870346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:37.872935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:38.080961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 825/1644] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:37 2022] POST /identity/v3/projects => generated 380 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:32:38.081162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:38.102413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fcd9a-d012-4251-9220-41bb4290be20 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:38.277255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:38.278292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:38.278737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:38.279172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:38.279873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:38.282077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:38.502387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 820/1645] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:38 2022] POST /identity/v3/users => generated 291 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:38.511625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:38.524863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf5f86-fb67-41b2-88ad-b74e79236968 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:38.532691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:38.534119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:38.534337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:38.534796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:38.536065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:38.539048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:38.558218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:38.573256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 826/1646] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:38 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:38.574584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc04d-085a-41a3-bc9b-aa1dcc19e7c8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:38.581544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:38.582747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:38.583572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:38.584143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:38.585360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:38.588063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eea299-a100-46f4-9697-530dfb0bfaad admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:38.615905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:38.629765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 821/1647] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:38 2022] GET /identity/v3/roles => generated 1427 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:32:38.631631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f82a-5c5f-4d74-a246-1d07243beffc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:38.638030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:38.639291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:38.639576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:38.639852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] PATH_INFO: `/v3/projects/66fc791b89bc48baaec4a6b371d1c904/users/b7a2a5b259c648b3adb37aa1bfbb5334/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:38.660044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66fc791b89bc48baaec4a6b371d1c904, user_id=b7a2a5b259c648b3adb37aa1bfbb5334, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:38.663415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:38.770716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 827/1648] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:38 2022] PUT /identity/v3/projects/66fc791b89bc48baaec4a6b371d1c904/users/b7a2a5b259c648b3adb37aa1bfbb5334/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:38.785832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:38.786122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:38.786683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:38.865184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] MFA Rules not processed for user `b7a2a5b259c648b3adb37aa1bfbb5334`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:38.866432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:39.035912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:39.043892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 822/1649] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:39.229146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7317-1e3a-4338-8548-60a32a1a4c1c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:39.242206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:39.246019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:39.246543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:39.251748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:39.252826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:39.255038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:39.420011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 828/1650] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:39 2022] POST /identity/v3/projects => generated 380 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:39.425626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:39.445106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d26b8-c964-4930-add1-f6b5ad65b58d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:39.471248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:39.600690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:39.602115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:39.602623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:39.603356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:39.604264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:39.606709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:39.758606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 823/1651] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:39 2022] POST /identity/v3/users => generated 297 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:39.763852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:39.779282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353f03f-b33b-431d-b33c-cf670f96ebd2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:39.802681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:39.803332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:39.803616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:39.803890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:39.805392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:39.807117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:39.827246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 829/1652] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:39 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:39.831587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:39.846635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d15f7b-16ce-41a3-9c01-3e56c1d12160 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:39.859646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:39.863401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:39.863886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:39.864176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] PATH_INFO: `/v3/projects/2ba4c92eb36645c38cb8abce281caea5/users/802344efb45e43d19fe6c5138abc7d9a/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:39.890522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba4c92eb36645c38cb8abce281caea5, user_id=802344efb45e43d19fe6c5138abc7d9a, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:39.894883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:40.009154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 824/1653] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:39 2022] PUT /identity/v3/projects/2ba4c92eb36645c38cb8abce281caea5/users/802344efb45e43d19fe6c5138abc7d9a/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:40.010203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:40.027310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5339e-e7a0-4dc7-ab3b-28c9566b2568 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:40.062547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:40.137584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:40.138276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:40.140357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:40.140696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:40.141406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:40.143217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2f84f-41f9-445f-83da-1fab03844a8f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:40.156810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 830/1654] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:40 2022] GET /identity/v3/roles => generated 1427 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:40.161614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:40.175768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dec42-df3e-4b77-ac9c-8e7f057d9990 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:40.182004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:40.182796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:40.183157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:40.183509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] PATH_INFO: `/v3/projects/2ba4c92eb36645c38cb8abce281caea5/users/802344efb45e43d19fe6c5138abc7d9a/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:40.185937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba4c92eb36645c38cb8abce281caea5, user_id=802344efb45e43d19fe6c5138abc7d9a, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:40.188918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:40.206552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 825/1655] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:40 2022] PUT /identity/v3/projects/2ba4c92eb36645c38cb8abce281caea5/users/802344efb45e43d19fe6c5138abc7d9a/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:40.217181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fff62-f356-4a1a-8b50-379c049d95fa None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:40.217475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fff62-f356-4a1a-8b50-379c049d95fa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:40.217755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fff62-f356-4a1a-8b50-379c049d95fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:40.252293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9b0fff62-f356-4a1a-8b50-379c049d95fa None None] MFA Rules not processed for user `802344efb45e43d19fe6c5138abc7d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:40.266574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9b0fff62-f356-4a1a-8b50-379c049d95fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:40.404628 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 831/1656] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:40.418374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:40.420687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:40.547382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51727370-f622-47d7-b2ed-ae14b5f2ea99 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:40.581171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:40.582167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:40.582748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:40.583192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:40.598955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:40.600556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:40.620658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 826/1657] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:32:40.859024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:40.860735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:40.862980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:40.913913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] MFA Rules not processed for user `4c6fffcc3c0d4c49848fa6c2f6b90130`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:40.916023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:40.981937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:41.000753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a61ddd-3ecc-482d-b43b-3623acdc182c service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:41.018109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:41.019616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:41.022996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:41.023533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:41.071351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 832/1658] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:41.087172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:41.088985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:41.156837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:41.164025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:41.189696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 827/1659] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:32:41.305125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bc6ec-dc96-4303-9e26-8f92a402a04f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:41.312976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:41.314078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:41.315085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:41.315517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:41.335310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:41.337237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d5422-dd6e-44cc-879d-0a6b187e9622 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:41.360573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 833/1660] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:32:42.148438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:44.034789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:44.036251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:44.045972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:44.058302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189015c1-6e50-48ca-854a-a87c84678408 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:44.067285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:44.068134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:44.071352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:44.075647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:44.077458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:44.357749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:44.359230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79496a4a-79e7-4b42-a41c-4345dc1647b6 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:44.396637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 828/1661] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:45.069921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:45.070766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:45.071187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:45.111029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] MFA Rules not processed for user `07e70b1c968443eab469777283b8fb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:45.115944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:45.297615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 834/1662] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:45.308367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8cb4-4b2e-4ac2-a473-0fe72f2f251b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:45.308822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8cb4-4b2e-4ac2-a473-0fe72f2f251b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:45.309239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d8cb4-4b2e-4ac2-a473-0fe72f2f251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:45.335077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-313d8cb4-4b2e-4ac2-a473-0fe72f2f251b None None] MFA Rules not processed for user `07e70b1c968443eab469777283b8fb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:45.336899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-313d8cb4-4b2e-4ac2-a473-0fe72f2f251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:45.357529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 829/1663] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:45.375971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:45.404842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b856e-c0e7-4efc-9ccb-de443b9939ef None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:45.415431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:45.416724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:45.417023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:45.417287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:45.437118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:45.439363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72701bff-96a3-41c5-9d44-46861b1609fe service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:45.485904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 835/1664] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:47.183543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c8d2-3872-493c-969f-5fa268774a08 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.183957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c8d2-3872-493c-969f-5fa268774a08 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.184126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442c8d2-3872-493c-969f-5fa268774a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.301560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c442c8d2-3872-493c-969f-5fa268774a08 None None] MFA Rules not processed for user `1b9d76409f834b478981c0bbc0c3d4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:47.302871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c442c8d2-3872-493c-969f-5fa268774a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.380162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.381384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.382173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.465158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] MFA Rules not processed for user `db18d0211f2a4e1c8949726ded7a9bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:47.466805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.468726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 830/1665] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:47.491401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf70a34-e1ea-43a0-aa00-f0a786a946a7 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.497833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf70a34-e1ea-43a0-aa00-f0a786a946a7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.497833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf70a34-e1ea-43a0-aa00-f0a786a946a7 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.500260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 831/1666] 10.222.0.144 () {54 vars in 917 bytes} [Tue Jul 26 16:32:47 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:47.507492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051f8a-873a-431f-8f31-0d6ae96d640d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.508046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051f8a-873a-431f-8f31-0d6ae96d640d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.509806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051f8a-873a-431f-8f31-0d6ae96d640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.614798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b7051f8a-873a-431f-8f31-0d6ae96d640d None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:47.617248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b7051f8a-873a-431f-8f31-0d6ae96d640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.673666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 836/1667] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:47.673666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:47.673666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.695782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26af1-87db-4137-a4f4-a57e30ee1586 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:47.704210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:47.705592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.705856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 832/1668] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:47.706478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.707033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.707158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.707312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.707579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.722778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:47.726216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:47.738908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] MFA Rules not processed for user `db18d0211f2a4e1c8949726ded7a9bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:47.739820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 837/1669] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:47.740127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.742677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:47.744883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:47.754202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 833/1670] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:47.763632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51458-8c95-4232-994b-2ed6efed861a service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:47.773733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:47.774995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:47.775470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:47.776065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:47.782797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:47.786670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:47.788715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742d544-e23f-44ec-bb21-83a9639bf394 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:47.801382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 838/1671] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:48.000284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:48.014586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e61dac-1e22-4a9a-b855-457f4581d266 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:48.021338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:48.022821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.023316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.023780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.033946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:48.037500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf981c-4268-405c-b386-9cfefeeb3967 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:48.045845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:48.078923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 834/1672] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:48.666262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.666757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.667012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.668481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 839/1673] 10.222.0.144 () {54 vars in 917 bytes} [Tue Jul 26 16:32:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:48.683874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0742fe-bb71-4c32-ab92-fd5030be760f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.689762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0742fe-bb71-4c32-ab92-fd5030be760f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.690302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0742fe-bb71-4c32-ab92-fd5030be760f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.716924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fe0742fe-bb71-4c32-ab92-fd5030be760f None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:48.718241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fe0742fe-bb71-4c32-ab92-fd5030be760f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:48.728505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 835/1674] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:48.752147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:48.755279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:48.781627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af6741-b90d-42de-a911-7a1ced47abea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:48.786234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:48.787071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.787500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.787765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.821702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:48.821910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.823781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69733ee-4e0f-469e-8509-b9aef51b261b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:48.825032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.825315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.834744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 840/1675] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:48.846654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe0fe-8272-4bba-921a-d0fc2f946162 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.846654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe0fe-8272-4bba-921a-d0fc2f946162 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.846654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe0fe-8272-4bba-921a-d0fc2f946162 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.852141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 841/1676] 10.222.0.144 () {54 vars in 966 bytes} [Tue Jul 26 16:32:48 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:32:48.853506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2d4f-edc8-45a7-ac6a-88400e04c40c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:48.853788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2d4f-edc8-45a7-ac6a-88400e04c40c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:48.854057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2d4f-edc8-45a7-ac6a-88400e04c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:48.866884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] MFA Rules not processed for user `4c6fffcc3c0d4c49848fa6c2f6b90130`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:48.868937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:48.878568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 836/1677] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:48.890609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-657f2d4f-edc8-45a7-ac6a-88400e04c40c None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:48.892310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-657f2d4f-edc8-45a7-ac6a-88400e04c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:48.996310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:49.006686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 842/1678] 10.222.0.144 () {58 vars in 1063 bytes} [Tue Jul 26 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:49.012196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095418f-de15-491e-9044-93c4a6a6215b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:49.030056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:49.030056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:49.030056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:49.030056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:49.030763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:49.045771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0b3d4-a021-4fb9-b019-cbb269cb9b37 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:49.049509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0b3d4-a021-4fb9-b019-cbb269cb9b37 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:49.049509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0b3d4-a021-4fb9-b019-cbb269cb9b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:49.076725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:49.079869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:49.086362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6fb0b3d4-a021-4fb9-b019-cbb269cb9b37 None None] MFA Rules not processed for user `07e70b1c968443eab469777283b8fb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:49.089358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6fb0b3d4-a021-4fb9-b019-cbb269cb9b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:49.108756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 843/1679] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:32:49.122410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 837/1680] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:54.411373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316a7a-ac94-4863-900f-3ae24f19c88b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:54.412680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316a7a-ac94-4863-900f-3ae24f19c88b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:54.413215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316a7a-ac94-4863-900f-3ae24f19c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:54.460466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:54.464857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:54.502711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ea316a7a-ac94-4863-900f-3ae24f19c88b None None] MFA Rules not processed for user `1b9d76409f834b478981c0bbc0c3d4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:54.506142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea316a7a-ac94-4863-900f-3ae24f19c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:54.540871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 844/1681] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:54.672165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7088-9818-40af-af2c-00599a744255 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:54.688411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:54.691421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:54.691884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:54.692344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] PATH_INFO: `/v3/users/dfa6a4b037964f22b6f9ee25acf6a094` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:54.698860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa6a4b037964f22b6f9ee25acf6a094)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:54.705085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:55.087311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa6a4b037964f22b6f9ee25acf6a094'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:55.088748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa6a4b037964f22b6f9ee25acf6a094'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:55.226806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa6a4b037964f22b6f9ee25acf6a094'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:55.228321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa6a4b037964f22b6f9ee25acf6a094'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:55.249149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa6a4b037964f22b6f9ee25acf6a094'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:55.292437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 838/1682] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:32:54 2022] DELETE /identity/v3/users/dfa6a4b037964f22b6f9ee25acf6a094 => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:32:55.297125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:55.298406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:55.298830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:55.329155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:55.330874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:55.447904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 845/1683] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:55.458432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:55.461627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:55.569525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30183-54eb-4f16-ab8c-f7d06d24a354 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:55.582748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:55.586545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:55.587349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:55.588508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:55.601085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:55.606980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:55.608480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:55.630414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 839/1684] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:56.365622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:56.367213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:56.386864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa08ff-ef72-4f5c-9b9d-79fc802c2fc2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:56.409926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:56.410597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:56.410876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:56.411146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] PATH_INFO: `/v3/projects/14289132bbb4432f9afaa3376835eb8e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:56.412327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] RBAC: Authorizing `identity:delete_project(project_id=14289132bbb4432f9afaa3376835eb8e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:56.414293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:56.599226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14289132bbb4432f9afaa3376835eb8e', 'request_id': 'req-423bf179-9793-48a7-87ba-6b6eb9c007df'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:56.611956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14289132bbb4432f9afaa3376835eb8e', 'request_id': 'req-423bf179-9793-48a7-87ba-6b6eb9c007df'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:56.855987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14289132bbb4432f9afaa3376835eb8e', 'request_id': 'req-423bf179-9793-48a7-87ba-6b6eb9c007df'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:56.861955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:32:56.881415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] The token cache is being invalidate because project 14289132bbb4432f9afaa3376835eb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:32:56.881964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:56.893615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-423bf179-9793-48a7-87ba-6b6eb9c007df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:56.893615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 846/1685] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:32:56 2022] DELETE /identity/v3/projects/14289132bbb4432f9afaa3376835eb8e => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:56.899390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca0760-77a5-43c0-bc71-7401e80ab4cc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:56.901658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:56.902257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:56.902374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca0760-77a5-43c0-bc71-7401e80ab4cc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:56.902374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca0760-77a5-43c0-bc71-7401e80ab4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:56.921010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0811-d4cf-47df-a7e6-fc34550e766d service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:56.944371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a5ca0760-77a5-43c0-bc71-7401e80ab4cc None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:56.950380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5ca0760-77a5-43c0-bc71-7401e80ab4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:57.016015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.016714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.017289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.017711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] PATH_INFO: `/v3/users/10dd7a84ac7b4cde9bef32976f374386` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.019171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=10dd7a84ac7b4cde9bef32976f374386)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.021357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.026376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 847/1686] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:57.035962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.036355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.036729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.074273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:57.075698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:57.091003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 848/1687] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:57.091914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.101763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbcf8-f123-424f-b864-faacf4d985ce None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.106778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.107462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.107743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.108013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.108745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:57.109044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.110763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.126830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 849/1688] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:57.132623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.146026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92567a2c-ed99-4421-8ce8-49e4325da078 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.151241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.152020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.152421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.152830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.153604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.155665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.463039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 850/1689] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:57 2022] POST /identity/v3/projects => generated 404 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:32:57.467265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.527221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b82bbc-2603-4950-a02c-fcc503370ef1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.531791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.532406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.532701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.532975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.533631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.536087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.557452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10dd7a84ac7b4cde9bef32976f374386'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:57.558125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10dd7a84ac7b4cde9bef32976f374386'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:57.710354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10dd7a84ac7b4cde9bef32976f374386'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:57.711557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10dd7a84ac7b4cde9bef32976f374386'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:57.718344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 851/1690] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:57 2022] POST /identity/v3/users => generated 303 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:32:57.723851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.725902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:57.742061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560456c0-991d-4ee9-a27e-f35afe07b2a7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.745843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78e3d77d-249a-49de-95ed-c957522553ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10dd7a84ac7b4cde9bef32976f374386'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:57.747514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.749595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.750103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.750564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.751571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.754031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.772494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 852/1691] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:57 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:57.776005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.790456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c071e9-fd48-4bf8-a982-b9e7a2919516 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.799024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.799985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.800329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.800835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.803033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.804748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:57.807242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 840/1692] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:32:56 2022] DELETE /identity/v3/users/10dd7a84ac7b4cde9bef32976f374386 => generated 0 bytes in 909 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:57.813943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.814335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.814736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.826251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 853/1693] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:57 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:32:57.832577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.848879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:57.851096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:57.857653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233987d-2fb8-4f9d-bb8e-2feb1aee3331 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:57.937750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:57.938731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 841/1694] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:57.941029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:57.941630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:57.942055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] PATH_INFO: `/v3/projects/22ab647e5a7e4b81beed4fe0f16bb575/users/c54ac132cc2a4556839bd00af53c43e5/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:57.961551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:57.963507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:57.969381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ab647e5a7e4b81beed4fe0f16bb575, user_id=c54ac132cc2a4556839bd00af53c43e5, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:57.973079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:58.042135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35175a-f456-4fa9-8d9e-b2829e5eccfb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:58.047227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:58.047227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:58.047227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:58.047227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:58.057369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:58.059087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7870-bdb3-4eb9-8151-be0d17f14e84 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:58.069392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 842/1695] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:58.134466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 854/1696] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:57 2022] PUT /identity/v3/projects/22ab647e5a7e4b81beed4fe0f16bb575/users/c54ac132cc2a4556839bd00af53c43e5/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:58.136637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:58.136927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:58.137198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:58.158496 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:58.159707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:58.242209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 843/1697] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:58.252803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:58.254804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:58.331492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3335e6-a2ac-4986-9433-e1f28adee60a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:58.335488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:58.336253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:58.337381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:58.337788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:58.349458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:58.351163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8622f1-415a-40cf-a205-6a3acfdee711 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:58.365646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 855/1698] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:32:58.624237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:58.626165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:58.646439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eb933-29ff-459e-a0a4-cca33bfd6fef None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:58.651208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:58.651862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:58.652143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:58.652450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] PATH_INFO: `/v3/projects/23bea48eb25e4818bce5b81884348840` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:58.653618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23bea48eb25e4818bce5b81884348840)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:58.654633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: WARNING py.warnings [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:32:58.654633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: warnings.warn(msg) Jul 26 16:32:58.654633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: Jul 26 16:32:58.655823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:58.896605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23bea48eb25e4818bce5b81884348840', 'request_id': 'req-b49e8733-ee43-44b3-804b-f48aaf30af91'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:58.910401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23bea48eb25e4818bce5b81884348840', 'request_id': 'req-b49e8733-ee43-44b3-804b-f48aaf30af91'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:58.916599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23bea48eb25e4818bce5b81884348840', 'request_id': 'req-b49e8733-ee43-44b3-804b-f48aaf30af91'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:59.020313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] The token cache is being invalidate because project 23bea48eb25e4818bce5b81884348840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:32:59.021327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:59.022151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e8733-ee43-44b3-804b-f48aaf30af91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:32:59.028640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 844/1699] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:32:58 2022] DELETE /identity/v3/projects/23bea48eb25e4818bce5b81884348840 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:32:59.036125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.036720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.036934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.061731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:59.063418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:59.158631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 856/1700] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:59.166933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.167325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.167710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.187776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:59.189041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:59.197556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 845/1701] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:59.201818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.213886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf52981-7f44-4658-a56e-cdca3318ad15 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.218428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.220477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.220477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.220477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.220477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:32:59.220763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.222289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.233726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 857/1702] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:32:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:32:59.238109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.247196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a740ca-0103-466c-ac3b-4cd46f01ac24 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.251770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.252374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.252683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.253473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.254180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.256088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.429837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 846/1703] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/projects => generated 406 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:32:59.433883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.445170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda943f-7483-452a-b6bd-83f61ce37c45 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.513919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.514806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.515090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.515368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.516035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.518088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.643450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 858/1704] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/users => generated 304 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:32:59.648803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.659345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871d95b-ecc6-4caa-bb02-5399bebc090e admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.664118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.664763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.665049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.665326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.666057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.667650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.679918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 847/1705] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:59 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:59.687009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.698522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e6818-bb2b-41cc-ab29-6003c34d725f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.703117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.704086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.704370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.704674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.705805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.706962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cf687-5f5f-4a80-a554-365a8b433dd5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.727288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 859/1706] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:32:59 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:32:59.728481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:32:59.739532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188bcc-d4a1-4a71-bf2a-dc555b09d1b8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:32:59.745379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:32:59.746055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.746336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.746608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] PATH_INFO: `/v3/projects/972b28c33ff14091823f9211a0fe926e/users/4aabc79db7b34e2ba226d2d67186a00d/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.765173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=972b28c33ff14091823f9211a0fe926e, user_id=4aabc79db7b34e2ba226d2d67186a00d, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:32:59.767991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:32:59.888723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 848/1707] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:32:59 2022] PUT /identity/v3/projects/972b28c33ff14091823f9211a0fe926e/users/4aabc79db7b34e2ba226d2d67186a00d/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:32:59.900456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:32:59.900750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:32:59.901052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:32:59.930261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] MFA Rules not processed for user `4aabc79db7b34e2ba226d2d67186a00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:32:59.931477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:32:59.995363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 860/1708] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:32:59.999914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.078124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7608e-b562-4f1b-8a94-649f3e220f3b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:00.082899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:00.083811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.084461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.084874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.085562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:00.087501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbbb3f-80dd-4938-87b2-3f15673d2c2f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:00.208576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 849/1709] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:32:59 2022] POST /identity/v3/projects => generated 406 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:00.209474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.234950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e933b-1ae7-4c25-a077-7cba0f382393 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:00.326547 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:00.346504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.346504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.346504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.346504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:00.346504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:00.503748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.509392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.509392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.550522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 861/1710] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:00 2022] POST /identity/v3/users => generated 310 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:00.557820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.565932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] MFA Rules not processed for user `c54ac132cc2a4556839bd00af53c43e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:00.567204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:00.579769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7619d4-c700-42af-bc71-7d000a957d39 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:00.590405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:00.591212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.591562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.591904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.605610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:00.611277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:00.630513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 862/1711] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:00 2022] GET /identity/v3/roles => generated 1427 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:00.638682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.659361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5631da-4639-4455-bbd8-9fa32f20e61f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:00.672284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:00.673004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.673291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.673571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] PATH_INFO: `/v3/projects/a188fdb9a79e4ea1bec0865e36537509/users/dce5bc8ee16d47838eb1a2e1ffb49200/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.687920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 850/1712] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:00.687920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.687920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:00.714035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecadaf8-3809-41f8-a7e5-ac30a787d12c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:00.718250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188fdb9a79e4ea1bec0865e36537509, user_id=dce5bc8ee16d47838eb1a2e1ffb49200, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:00.725696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:00.730659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:00.736883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:00.736883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:00.736883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:00.740341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:00.742658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:00.936842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 863/1713] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:00 2022] PUT /identity/v3/projects/a188fdb9a79e4ea1bec0865e36537509/users/dce5bc8ee16d47838eb1a2e1ffb49200/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:00.947318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:00.963823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 851/1714] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:00 2022] POST /identity/v3/projects => generated 402 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:00.969241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.065978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdd604-70f1-4ca1-a4f2-cfd5be9b21ea admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.070036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3425-0c78-4ad8-9a4f-14b222740496 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.074292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.075307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.075801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.076480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.078115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.082174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.090775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.092266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.092747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.093177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.094601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.098271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.122260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 864/1715] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:00 2022] GET /identity/v3/roles => generated 1427 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:01.124186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.140580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3239-b709-43bd-8e24-cfe485f6665a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.158107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.160047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.160047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.160047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] PATH_INFO: `/v3/projects/a188fdb9a79e4ea1bec0865e36537509/users/dce5bc8ee16d47838eb1a2e1ffb49200/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.167372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188fdb9a79e4ea1bec0865e36537509, user_id=dce5bc8ee16d47838eb1a2e1ffb49200, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.170145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ece891-e839-4474-9298-9b5da219ac9a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.225490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 865/1716] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:01 2022] PUT /identity/v3/projects/a188fdb9a79e4ea1bec0865e36537509/users/dce5bc8ee16d47838eb1a2e1ffb49200/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:01.228130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.228695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.229214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.284464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] MFA Rules not processed for user `dce5bc8ee16d47838eb1a2e1ffb49200`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:01.286076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:01.323237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 852/1717] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:00 2022] POST /identity/v3/users => generated 308 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:01.326627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.345928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525aaec-c979-4451-8b42-5986612fda66 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.400201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 866/1718] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:01.410534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.414859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:01.469862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.470569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.470865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.471148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.473073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.478520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.499679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 853/1719] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:01 2022] GET /identity/v3/roles => generated 1427 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:01.504179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.515381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9f26b-0146-4d02-811a-a79f5a569613 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.519306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f41d7-9c7e-411b-b9ef-52a1fadf1b0f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.523139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.527841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.527841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.527841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.529539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.531544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.531835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.532135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] PATH_INFO: `/v3/projects/53a7b6f18d324fa097629bc60d848b6b/users/769ca21ad767443ca8661fe02470ce71/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.538154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.539896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac685f80-93b5-4b6c-be34-62813a123abd service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.552745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 867/1720] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:01.564298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a7b6f18d324fa097629bc60d848b6b, user_id=769ca21ad767443ca8661fe02470ce71, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.567064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.600135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:01.652860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.653157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.653479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.685096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 854/1721] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:01 2022] PUT /identity/v3/projects/53a7b6f18d324fa097629bc60d848b6b/users/769ca21ad767443ca8661fe02470ce71/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:01.688795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.734493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:01.735857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:01.822573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ed93c-816c-4af4-89b1-6f7931048030 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.833766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.834394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.834679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.834954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.836798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.838429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.865763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 868/1722] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:01.874308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 855/1723] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:01 2022] GET /identity/v3/roles => generated 1427 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:01.880654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:01.901559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc12b7-940f-4fa3-9700-16004ec5d9a1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:01.907436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:01.908765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:01.909382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:01.909832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] PATH_INFO: `/v3/projects/53a7b6f18d324fa097629bc60d848b6b/users/769ca21ad767443ca8661fe02470ce71/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:01.912209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a7b6f18d324fa097629bc60d848b6b, user_id=769ca21ad767443ca8661fe02470ce71, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:01.916320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:01.931976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 869/1724] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:01 2022] PUT /identity/v3/projects/53a7b6f18d324fa097629bc60d848b6b/users/769ca21ad767443ca8661fe02470ce71/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:02.266572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:02.269176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:02.376607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430e6d5-bf7d-4784-b133-4ac3304fa45f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:02.460618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:02.461261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:02.461545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:02.461826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:02.552425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:02.554942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971ed5-9936-425a-aa24-fc12afee4f79 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:02.575916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 856/1725] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:33:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:33:02.646100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:04.332186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:04.407125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b282-df67-4f68-b13a-f9229d3ea9e7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:04.413228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:04.414020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:04.414400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:04.414762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] PATH_INFO: `/v3/users/4aabc79db7b34e2ba226d2d67186a00d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:04.416152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aabc79db7b34e2ba226d2d67186a00d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:04.417673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 16:33:04.417673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 26 16:33:04.417673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: Jul 26 16:33:04.419235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:04.975569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aabc79db7b34e2ba226d2d67186a00d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:04.976615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aabc79db7b34e2ba226d2d67186a00d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:04.990742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aabc79db7b34e2ba226d2d67186a00d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:04.991635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aabc79db7b34e2ba226d2d67186a00d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:05.021310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7de263ef-527d-4845-87c3-9851f014862c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aabc79db7b34e2ba226d2d67186a00d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:05.153952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 870/1726] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:04 2022] DELETE /identity/v3/users/4aabc79db7b34e2ba226d2d67186a00d => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:05.159646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:05.160189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:05.160649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:05.183817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:05.185233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:05.254762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 857/1727] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:05.267650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012c3-f3dc-4c99-b248-ce531a4362ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:05.268465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012c3-f3dc-4c99-b248-ce531a4362ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:05.268465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012c3-f3dc-4c99-b248-ce531a4362ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:05.274858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:05.276263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:05.304718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-976012c3-f3dc-4c99-b248-ce531a4362ea None None] MFA Rules not processed for user `769ca21ad767443ca8661fe02470ce71`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:05.305492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-976012c3-f3dc-4c99-b248-ce531a4362ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:05.334112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d813b-0a8a-4c1f-8151-01f48ddc0b48 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:05.338943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:05.338943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:05.338943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:05.339239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:05.348737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:05.350155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:05.359592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 858/1728] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:05.376083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 871/1729] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:05 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:06.045257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:06.046861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:06.077505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8e76-d4a8-4822-9a74-73d9417d7b47 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:06.083959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:06.085038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:06.085516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:06.085883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] PATH_INFO: `/v3/projects/972b28c33ff14091823f9211a0fe926e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:06.087461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=972b28c33ff14091823f9211a0fe926e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:06.089557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:06.502362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee531-3f66-49d8-ab00-07866066dc9b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:06.504418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee531-3f66-49d8-ab00-07866066dc9b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:06.505064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee531-3f66-49d8-ab00-07866066dc9b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:06.508206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 872/1730] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:33:06 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:33:06.529911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:06.530377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:06.530759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:06.558720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:06.560339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:06.626478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972b28c33ff14091823f9211a0fe926e', 'request_id': 'req-865bdf88-f02e-48d5-b8c3-760dca35dcc3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:06.645237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972b28c33ff14091823f9211a0fe926e', 'request_id': 'req-865bdf88-f02e-48d5-b8c3-760dca35dcc3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:06.654074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972b28c33ff14091823f9211a0fe926e', 'request_id': 'req-865bdf88-f02e-48d5-b8c3-760dca35dcc3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:06.681455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 873/1731] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:33:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:06.693378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:06.694852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:06.708534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5af26-fb90-4cb9-b01f-9a282ac611a4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:06.851189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:06.852003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:06.852575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:06.852958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:06.863423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:06.865981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:06.880055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] The token cache is being invalidate because project 972b28c33ff14091823f9211a0fe926e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:06.881168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:06.882106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-865bdf88-f02e-48d5-b8c3-760dca35dcc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:06.883511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 874/1732] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:06.885924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 859/1733] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:06 2022] DELETE /identity/v3/projects/972b28c33ff14091823f9211a0fe926e => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:06.894138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:06.992169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aadcd-87c2-4306-ba8e-91be5e9af0e1 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:06.997661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:06.998919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:06.998919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:06.999167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] PATH_INFO: `/v3/users/dce5bc8ee16d47838eb1a2e1ffb49200` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:07.000193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce5bc8ee16d47838eb1a2e1ffb49200)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:07.002217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:07.260305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce5bc8ee16d47838eb1a2e1ffb49200'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:07.261154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce5bc8ee16d47838eb1a2e1ffb49200'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:07.293749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce5bc8ee16d47838eb1a2e1ffb49200'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:07.295724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce5bc8ee16d47838eb1a2e1ffb49200'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:07.339010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39af272f-66e5-4f14-a01c-e6e4db0a46a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce5bc8ee16d47838eb1a2e1ffb49200'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:07.456609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 875/1734] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:06 2022] DELETE /identity/v3/users/dce5bc8ee16d47838eb1a2e1ffb49200 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:07.684375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:07.910541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:07.912749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:07.912749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:07.913098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 860/1735] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:33:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:33:07.943519 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:07.944060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:07.944060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:07.988941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] MFA Rules not processed for user `8c7f8c220a3844469072a97f7d2cb6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:07.990158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:08.071118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 876/1736] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:08.083993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:08.086266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:08.105296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93459-f29d-49d8-9f83-0f84a935b922 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:08.110369 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:08.110993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:08.111273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:08.111545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:08.121559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:08.123498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:08.136955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 861/1737] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:08.191895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:08.193573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:08.223100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:08.224785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:08.324082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae565f-4260-4c69-a64e-2339fe591a9d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:08.331003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:08.331694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:08.331963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:08.332267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] PATH_INFO: `/v3/users/c54ac132cc2a4556839bd00af53c43e5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:08.333610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54ac132cc2a4556839bd00af53c43e5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:08.336294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:08.384841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bdcce-b577-4fce-816b-c3e4327ca6d7 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:08.391244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:08.392321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:08.392643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:08.392944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] PATH_INFO: `/v3/projects/a188fdb9a79e4ea1bec0865e36537509` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:08.394532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a188fdb9a79e4ea1bec0865e36537509)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:08.396963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:08.658345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a188fdb9a79e4ea1bec0865e36537509', 'request_id': 'req-97adbb0b-5b19-4a9a-aafd-42a233c89746'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.671808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a188fdb9a79e4ea1bec0865e36537509', 'request_id': 'req-97adbb0b-5b19-4a9a-aafd-42a233c89746'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.673736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54ac132cc2a4556839bd00af53c43e5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.674647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54ac132cc2a4556839bd00af53c43e5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.686949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a188fdb9a79e4ea1bec0865e36537509', 'request_id': 'req-97adbb0b-5b19-4a9a-aafd-42a233c89746'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.688947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54ac132cc2a4556839bd00af53c43e5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.690199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54ac132cc2a4556839bd00af53c43e5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.707866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a548d004-5d02-4c2d-ba0a-b33d7e4f1172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54ac132cc2a4556839bd00af53c43e5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.839883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] The token cache is being invalidate because project a188fdb9a79e4ea1bec0865e36537509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:08.840555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.841074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-97adbb0b-5b19-4a9a-aafd-42a233c89746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:08.842756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 862/1738] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:08 2022] DELETE /identity/v3/projects/a188fdb9a79e4ea1bec0865e36537509 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:08.848563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:08.849412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:08.849701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:08.873938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:08.875040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:08.938275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 877/1739] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:08 2022] DELETE /identity/v3/users/c54ac132cc2a4556839bd00af53c43e5 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:08.975521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 863/1740] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:08.975974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:08.976535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:08.976642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.000635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:09.001973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:09.138352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 878/1741] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:09.141291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.154592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e79df-dae2-49e5-aa93-d36d00ce8ddc None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.161650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.162668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.163148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.163577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.164797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:09.165346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.168088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.190043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 864/1742] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:09.194029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.204984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a22ddd-a342-402b-89e8-af1431cf953f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.212103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.212799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.213085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.213644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.214493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.216465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.414099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 879/1743] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:09 2022] POST /identity/v3/projects => generated 388 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:09.417042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.432263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fee5e-b4b2-4cf2-91a2-4ce81fc536ec admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.439466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:09.541615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.542232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.542511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.542782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.543437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.545539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.708117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 865/1744] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:09 2022] POST /identity/v3/users => generated 295 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:09.709649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.712312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.714004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:09.725849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24454e31-cbe2-408e-89d4-d5746bf67d03 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.740884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.741903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.742202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.742592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.743393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.745909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.769319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948828-070c-4de0-8ccc-9f6fd48347e8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.777043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 880/1745] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:09 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:09.780491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.791086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.792041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.792506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.792941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] PATH_INFO: `/v3/projects/22ab647e5a7e4b81beed4fe0f16bb575` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.794439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35207-39ac-49de-a78e-85a82cf5aeae admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.795249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ab647e5a7e4b81beed4fe0f16bb575)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.797370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.811047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.819058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.819058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.819058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.819058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.819058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:09.840380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 881/1746] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:09 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:09.841554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:09.852566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c744fa-1bfd-4121-954d-3dc57c79030e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:09.858042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:09.858781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:09.859070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:09.859352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] PATH_INFO: `/v3/projects/abb10bbf7682446ea218b2d69e909779/users/39d139cbb8a94c0fb960e53c95b13676/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:09.874374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb10bbf7682446ea218b2d69e909779, user_id=39d139cbb8a94c0fb960e53c95b13676, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:09.877051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02672-6ef3-46df-85aa-a69e80feea95 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:10.047336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 882/1747] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:09 2022] PUT /identity/v3/projects/abb10bbf7682446ea218b2d69e909779/users/39d139cbb8a94c0fb960e53c95b13676/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:10.054075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ab647e5a7e4b81beed4fe0f16bb575', 'request_id': 'req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.060358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.062310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.062310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.062500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ab647e5a7e4b81beed4fe0f16bb575', 'request_id': 'req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.067139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ab647e5a7e4b81beed4fe0f16bb575', 'request_id': 'req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.088980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] MFA Rules not processed for user `39d139cbb8a94c0fb960e53c95b13676`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:10.090471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:10.159214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 883/1748] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:10.160475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:10.165235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] The token cache is being invalidate because project 22ab647e5a7e4b81beed4fe0f16bb575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:10.165891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.167249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.169487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 866/1749] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:09 2022] DELETE /identity/v3/projects/22ab647e5a7e4b81beed4fe0f16bb575 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:10.173442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:10.175675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:10.237019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a8b7b-1b5a-4532-a676-a4576554d26d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:10.253463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:10.254670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.255153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.255684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.256947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:10.260508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:10.282306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e9f1b-c56f-49f1-9d4a-22fcfb524eec admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:10.287913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:10.288704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.289059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.289404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] PATH_INFO: `/v3/users/769ca21ad767443ca8661fe02470ce71` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.291067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=769ca21ad767443ca8661fe02470ce71)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:10.293158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:10.399699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 884/1750] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:10 2022] POST /identity/v3/projects => generated 386 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:10.405424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:10.407300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:10.524087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127427e-9d66-42e4-8b18-4149544c983f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:10.551244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:10.552194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.552621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.552897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.553685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:10.553923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769ca21ad767443ca8661fe02470ce71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.554825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769ca21ad767443ca8661fe02470ce71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.555912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:10.677682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769ca21ad767443ca8661fe02470ce71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.678343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769ca21ad767443ca8661fe02470ce71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.685845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769ca21ad767443ca8661fe02470ce71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:10.704323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 867/1751] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:10 2022] DELETE /identity/v3/users/769ca21ad767443ca8661fe02470ce71 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:10.731457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 885/1752] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:10 2022] POST /identity/v3/users => generated 300 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:10.735459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:10.738007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:10.816659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e9f5-227e-4acd-9e97-c120530bdd62 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:10.822308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:10.836554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.836554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.836554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.836554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:10.836554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:10.837785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 868/1753] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:10 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:10.842298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:10.853451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d7a75-1c1d-44fb-97bf-df40251734c2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:10.870015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:10.870712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:10.870999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:10.871279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] PATH_INFO: `/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d/users/0d54ddd93ca34e63a81c597278822847/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:10.890844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99d1d22e0c449ecb7396ac74a66eb0d, user_id=0d54ddd93ca34e63a81c597278822847, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:10.894445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.090808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.092736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.093411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 886/1754] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:10 2022] PUT /identity/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d/users/0d54ddd93ca34e63a81c597278822847/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:11.097596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.111596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ab91-c071-4991-897d-ff7ce8ce3c65 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.162315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71077b8-3144-44bd-8f03-8342fe0392f9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.167200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.168059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.168305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.168610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.169302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.170897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.173819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.178098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.178098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.178098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] PATH_INFO: `/v3/projects/53a7b6f18d324fa097629bc60d848b6b` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.178098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a7b6f18d324fa097629bc60d848b6b)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.178098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.180725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 887/1755] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:11 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:11.184937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.194976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ceceb-d907-4378-9665-26e025f41d54 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.199950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.200645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.200934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.201375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] PATH_INFO: `/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d/users/0d54ddd93ca34e63a81c597278822847/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.204222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99d1d22e0c449ecb7396ac74a66eb0d, user_id=0d54ddd93ca34e63a81c597278822847, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.207600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9c6a8-ab2b-45ff-bf7f-d3582b756eda admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.225649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 888/1756] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:11 2022] PUT /identity/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d/users/0d54ddd93ca34e63a81c597278822847/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:11.229180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.229472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.229750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.257772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] MFA Rules not processed for user `0d54ddd93ca34e63a81c597278822847`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:11.260477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.312381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a7b6f18d324fa097629bc60d848b6b', 'request_id': 'req-e2194e23-adca-46ed-a2dd-76c9eeca57e3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.318988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a7b6f18d324fa097629bc60d848b6b', 'request_id': 'req-e2194e23-adca-46ed-a2dd-76c9eeca57e3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.324538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a7b6f18d324fa097629bc60d848b6b', 'request_id': 'req-e2194e23-adca-46ed-a2dd-76c9eeca57e3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.325450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 889/1757] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:11.337976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.339357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.397702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d5299-c6c3-47a6-b45c-c2ec2f8b3df1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.444361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.445004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.445299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.445562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.460978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] The token cache is being invalidate because project 53a7b6f18d324fa097629bc60d848b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:11.461410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.461914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.463454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 869/1758] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:11 2022] DELETE /identity/v3/projects/53a7b6f18d324fa097629bc60d848b6b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:11.464299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.465670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.511579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.511579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23345345-8881-44c2-a1fd-7fb843373341 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.521795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 890/1759] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:11.523153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae093-044b-420f-b81c-3c3dacaccc7f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.523463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae093-044b-420f-b81c-3c3dacaccc7f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.523736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cae093-044b-420f-b81c-3c3dacaccc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.534571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194e23-adca-46ed-a2dd-76c9eeca57e3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.539405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.540059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.540344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.540650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] PATH_INFO: `/v3/users/b7a2a5b259c648b3adb37aa1bfbb5334` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.541829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a2a5b259c648b3adb37aa1bfbb5334)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.543724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-16cae093-044b-420f-b81c-3c3dacaccc7f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:11.544878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.545019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-16cae093-044b-420f-b81c-3c3dacaccc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.555126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 891/1760] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:11.568488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.568488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.568488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.593783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:11.595079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.610732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 892/1761] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:11.610732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:11.616257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:11.625674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c02f3d-00ba-4588-a74e-97dd4ecd78b7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:11.631858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:11.632621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:11.632967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:11.633310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] PATH_INFO: `/v3/users/39d139cbb8a94c0fb960e53c95b13676` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:11.634568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d139cbb8a94c0fb960e53c95b13676)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:11.636635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:11.896608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a2a5b259c648b3adb37aa1bfbb5334'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.897272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a2a5b259c648b3adb37aa1bfbb5334'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.982483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d139cbb8a94c0fb960e53c95b13676'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.983180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d139cbb8a94c0fb960e53c95b13676'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.990502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d139cbb8a94c0fb960e53c95b13676'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:11.991053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d139cbb8a94c0fb960e53c95b13676'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:12.002386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d139cbb8a94c0fb960e53c95b13676'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:12.008750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a2a5b259c648b3adb37aa1bfbb5334'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:12.009530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a2a5b259c648b3adb37aa1bfbb5334'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:12.017067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a2a5b259c648b3adb37aa1bfbb5334'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:12.033801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 870/1762] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:11 2022] DELETE /identity/v3/users/b7a2a5b259c648b3adb37aa1bfbb5334 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:12.034580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.036451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.099831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd511a-7f1a-464e-8af8-10062819d0fb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.104404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.110272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38222a-5b2f-4e65-b033-282f100298fb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.123580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 871/1763] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:12.123580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.123580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.123580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.141673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:12.142874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.151249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 872/1764] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:12.152316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.162124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721a3a9-e836-4a86-9cc7-44c4db704be1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.166832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 893/1765] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:11 2022] DELETE /identity/v3/users/39d139cbb8a94c0fb960e53c95b13676 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:12.166832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.166832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.174168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.174800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.175085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.175359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.176151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.178148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.228811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fa886-69d9-4c3d-99ad-d424c33d0966 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.232590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.233227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.235279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.235279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.296582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.298003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219875b-9128-4df8-8df1-5fd313d2d407 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.307220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 894/1766] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:12.308435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.308707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.308980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.320376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 873/1767] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:12 2022] POST /identity/v3/projects => generated 356 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:12.328602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:12.329820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.330991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.392516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 895/1768] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:12.400019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77791290-01a4-4bd4-b9ad-d87ea6d8014b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.403951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.405446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.406604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.407366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.407690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.407965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.408784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.411832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.463480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa194fd-5ee3-4dcb-aa07-9b40d14fe825 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.467151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.467759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.468039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.468617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.480796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.482364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.493809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 896/1769] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:12.533558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 874/1770] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:12 2022] POST /identity/v3/users => generated 279 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:12.538360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.549582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd03c7-80dd-41c4-afaa-d39353052c43 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.563022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.563655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.563992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.564333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.565008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.569863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.593583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 897/1771] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:12 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:12.598924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.623059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b91762-aa94-4d24-bf9b-98b66e93f310 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.637304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.638792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.639680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.640345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.641295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.643638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012cb06-56ff-4f17-91c3-bb94bc99391f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.663671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 875/1772] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:12 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:12.668413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:12.694614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a30ced-4ba5-4b9a-8260-b06e84d53cfc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:12.699635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:12.700308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.700665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.700934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] PATH_INFO: `/v3/projects/1c721eb37361447cb1f210f83fb35ea7/users/86ee697e09de47b3b9c9045f1805e7d2/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.729582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c721eb37361447cb1f210f83fb35ea7, user_id=86ee697e09de47b3b9c9045f1805e7d2, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:12.732224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:12.815648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 898/1773] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:12 2022] PUT /identity/v3/projects/1c721eb37361447cb1f210f83fb35ea7/users/86ee697e09de47b3b9c9045f1805e7d2/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:12.827755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:12.828045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:12.828324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:12.864064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] MFA Rules not processed for user `86ee697e09de47b3b9c9045f1805e7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:12.865299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:12.995563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.008423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.008423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:13.014275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 876/1774] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:13.019031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.112918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f2e2d-61f8-4b6d-8113-cb5c183ce51d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.119361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.120498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.122153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.122153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] PATH_INFO: `/v3/projects/66fc791b89bc48baaec4a6b371d1c904` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.122970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66fc791b89bc48baaec4a6b371d1c904)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.125035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.128701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fc785-d2a4-4934-8742-f04e17ab5d50 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.133829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.134685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.135000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.135240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.135913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.137955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.304027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 877/1775] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:13 2022] POST /identity/v3/projects => generated 358 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:13.305339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.308175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.325073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:13.339725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66fc791b89bc48baaec4a6b371d1c904', 'request_id': 'req-fcc174a6-4362-44de-9f5b-02a2e71420f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.356067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66fc791b89bc48baaec4a6b371d1c904', 'request_id': 'req-fcc174a6-4362-44de-9f5b-02a2e71420f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.427967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d2df4-0e5f-4b8c-9f06-20410b93dc56 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.435681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.435681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.435681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.435681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] PATH_INFO: `/v3/projects/abb10bbf7682446ea218b2d69e909779` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.441559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb10bbf7682446ea218b2d69e909779)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.444072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.461499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66fc791b89bc48baaec4a6b371d1c904', 'request_id': 'req-fcc174a6-4362-44de-9f5b-02a2e71420f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.473905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] The token cache is being invalidate because project 66fc791b89bc48baaec4a6b371d1c904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:13.476235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.479032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.484119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 899/1776] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:12 2022] DELETE /identity/v3/projects/66fc791b89bc48baaec4a6b371d1c904 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:13.485302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.486922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.567170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc174a6-4362-44de-9f5b-02a2e71420f8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.571908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.572889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.573363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.573825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.575052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.578818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.611469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb10bbf7682446ea218b2d69e909779', 'request_id': 'req-8bd4a568-0a83-4317-bd83-731aaa5a86bb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.618263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb10bbf7682446ea218b2d69e909779', 'request_id': 'req-8bd4a568-0a83-4317-bd83-731aaa5a86bb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.626857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb10bbf7682446ea218b2d69e909779', 'request_id': 'req-8bd4a568-0a83-4317-bd83-731aaa5a86bb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.706792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 900/1777] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:13 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:13.708164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.709719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.721899 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4c3e-7846-4cf4-addb-f0747bc88cc4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.763988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] The token cache is being invalidate because project abb10bbf7682446ea218b2d69e909779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:13.764301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.765335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:13.770916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 878/1778] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:13 2022] DELETE /identity/v3/projects/abb10bbf7682446ea218b2d69e909779 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:13.771856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.773362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.788523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.789297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.789659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.790016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] PATH_INFO: `/v3/users/802344efb45e43d19fe6c5138abc7d9a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.791352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802344efb45e43d19fe6c5138abc7d9a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.793324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.860006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a568-0a83-4317-bd83-731aaa5a86bb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.867101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.867776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.868299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.868757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.869473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.871309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:13.885367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 879/1779] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:13 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:13.885367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:13.885367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:13.898658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9c7ab-c1d8-43ad-9618-fd2996b5c3f5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:13.903291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:13.903929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:13.904208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:13.904511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] PATH_INFO: `/v3/users/0d54ddd93ca34e63a81c597278822847` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:13.905685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d54ddd93ca34e63a81c597278822847)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:13.907623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:14.030260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802344efb45e43d19fe6c5138abc7d9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.030920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802344efb45e43d19fe6c5138abc7d9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.038391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802344efb45e43d19fe6c5138abc7d9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.039051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802344efb45e43d19fe6c5138abc7d9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.050912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802344efb45e43d19fe6c5138abc7d9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.249119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 901/1780] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:13 2022] DELETE /identity/v3/users/802344efb45e43d19fe6c5138abc7d9a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:14.250020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:14.251428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:14.286676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d54ddd93ca34e63a81c597278822847'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.286676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d54ddd93ca34e63a81c597278822847'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.336130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583522e-9c3c-4106-97b9-ba036275a691 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:14.342788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:14.345907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:14.345907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:14.345907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] PATH_INFO: `/v3/projects/51f963e7bae9451da70be719e34416ab/users/1af617629ce34eedbf9ff51ab9a77174/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:14.363524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f963e7bae9451da70be719e34416ab, user_id=1af617629ce34eedbf9ff51ab9a77174, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:14.366285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:14.466597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d54ddd93ca34e63a81c597278822847'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.467615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d54ddd93ca34e63a81c597278822847'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.481608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d54ddd93ca34e63a81c597278822847'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:14.485055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 902/1781] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:14 2022] PUT /identity/v3/projects/51f963e7bae9451da70be719e34416ab/users/1af617629ce34eedbf9ff51ab9a77174/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:14.493717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:14.501038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:14.531545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 880/1782] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:13 2022] DELETE /identity/v3/users/0d54ddd93ca34e63a81c597278822847 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:14.610106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4cc75-8e1d-47a7-a241-54d93dd95aa3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:14.690224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:14.691256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:14.691785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:14.692552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:14.693515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:14.695298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ead9-be89-4d35-9dfa-fbec6bbabae8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:14.707896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 903/1783] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:14 2022] GET /identity/v3/roles => generated 1427 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:14.712510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:14.726942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59f9c9-7078-4764-a8f0-ba22c4f52e08 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:14.733401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:14.734187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:14.741227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:14.741227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] PATH_INFO: `/v3/projects/51f963e7bae9451da70be719e34416ab/users/1af617629ce34eedbf9ff51ab9a77174/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:14.741227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f963e7bae9451da70be719e34416ab, user_id=1af617629ce34eedbf9ff51ab9a77174, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:14.741227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:14.753055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 881/1784] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:14 2022] PUT /identity/v3/projects/51f963e7bae9451da70be719e34416ab/users/1af617629ce34eedbf9ff51ab9a77174/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:14.763201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:14.763436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:14.763773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:14.812122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] MFA Rules not processed for user `1af617629ce34eedbf9ff51ab9a77174`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:14.813639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:14.926043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 904/1785] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:14.951479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:14.953143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:15.039131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5b872-251c-4752-a7ff-aad3252beb68 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:15.043832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:15.044511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:15.044786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:15.045066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:15.057344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:15.061266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:15.089152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 882/1786] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:15.096356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:15.096356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:15.210213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87771c6c-b8a6-4696-a2b6-987c0682e376 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:15.218078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:15.219017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:15.219511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:15.219868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] PATH_INFO: `/v3/projects/2ba4c92eb36645c38cb8abce281caea5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:15.221400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba4c92eb36645c38cb8abce281caea5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:15.225457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:16.325543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:16.338553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:16.347336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeec39-4ff9-431e-9f4f-3fe30036199b service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:16.364349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba4c92eb36645c38cb8abce281caea5', 'request_id': 'req-a762170b-8a3b-4544-b571-f81ee00ee139'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.373000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba4c92eb36645c38cb8abce281caea5', 'request_id': 'req-a762170b-8a3b-4544-b571-f81ee00ee139'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.428704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:16.429435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:16.429896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:16.430178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] PATH_INFO: `/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:16.431368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99d1d22e0c449ecb7396ac74a66eb0d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:16.433351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:16.671947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba4c92eb36645c38cb8abce281caea5', 'request_id': 'req-a762170b-8a3b-4544-b571-f81ee00ee139'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.688701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] The token cache is being invalidate because project 2ba4c92eb36645c38cb8abce281caea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:16.690677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.694550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.702019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 905/1787] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:15 2022] DELETE /identity/v3/projects/2ba4c92eb36645c38cb8abce281caea5 => generated 0 bytes in 1605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:16.702019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:16.702751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:16.713288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99d1d22e0c449ecb7396ac74a66eb0d', 'request_id': 'req-e7340e33-0231-4934-ab19-a7977df17343'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.723084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99d1d22e0c449ecb7396ac74a66eb0d', 'request_id': 'req-e7340e33-0231-4934-ab19-a7977df17343'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.727344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99d1d22e0c449ecb7396ac74a66eb0d', 'request_id': 'req-e7340e33-0231-4934-ab19-a7977df17343'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.783009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762170b-8a3b-4544-b571-f81ee00ee139 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:16.792983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:16.835308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:16.835938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:16.836219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:16.836517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:16.838351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:16.883915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] The token cache is being invalidate because project b99d1d22e0c449ecb7396ac74a66eb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:16.884300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.885121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e7340e33-0231-4934-ab19-a7977df17343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:16.886915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 883/1788] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:16 2022] DELETE /identity/v3/projects/b99d1d22e0c449ecb7396ac74a66eb0d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:16.888136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d17db-1240-49c4-a3b3-cac705c098b5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:16.888455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d17db-1240-49c4-a3b3-cac705c098b5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:16.888739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d17db-1240-49c4-a3b3-cac705c098b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:16.900085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:16.902121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82baabf-f69c-439f-942a-4dd4d86ab766 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:16.909105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-ba2d17db-1240-49c4-a3b3-cac705c098b5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:16.910589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ba2d17db-1240-49c4-a3b3-cac705c098b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:16.914488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 906/1789] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:16.915782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c85cf-0653-418b-97a8-59a8ee7e66a2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:16.916078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c85cf-0653-418b-97a8-59a8ee7e66a2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:16.916357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c85cf-0653-418b-97a8-59a8ee7e66a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:16.937514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-127c85cf-0653-418b-97a8-59a8ee7e66a2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:16.938820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-127c85cf-0653-418b-97a8-59a8ee7e66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:16.977110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 884/1790] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:16.986422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:16.986722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:16.986979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:16.998773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:17.006339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 907/1791] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:17.009562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:17.010906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:17.014198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.016149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.016149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.024534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 885/1792] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:17.028952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.037439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:17.037439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:17.041256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074eef-7afc-43d0-a2ef-fc70f4209702 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.046391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.049935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.049935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.049935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.049935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:17.049935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.050319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 908/1793] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:17.050807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.051287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.062129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6731-790d-49c9-8195-97c7580da2c4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.074249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 886/1794] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:17.074407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.074407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.074407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.074407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.074724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:17.074724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.074724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.075339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.085034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 909/1795] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:17.085766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efac56-b66f-46ba-9b1e-8d67afd07b5a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.089382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.091117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.091741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.092026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.092303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.093035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.094958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.106543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359677-5d8a-4dce-b725-2ce7279ff3f1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.106543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.106807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.107131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.107412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.108084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.110060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.240285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 910/1796] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/projects => generated 382 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:17.240516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 887/1797] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/projects => generated 372 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:17.241465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.244020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.304360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b97963-5345-4a8f-b6ff-a0f7656dfd2b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.304788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0db851-abeb-4a57-ac4b-90b7e6ade211 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.309257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.309719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.309876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.310160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.310379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.310486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.310691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.310985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.311183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.311691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.313134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.318466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.464119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 911/1798] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:17.467570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 888/1799] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:17.468042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.468162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.476959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42215f13-0f08-4b9d-9685-84d7cda8fdc5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.478696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e10ed-1056-4985-9909-385082a62ffe admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.483014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.483754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.484059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.484245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.484874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.484959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.485086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.485360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.486113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.487781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.497443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.499157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.499396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 912/1800] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:17.510785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.512468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 889/1801] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:17.516471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.523073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2ae7f-1a40-4154-94e8-bcd7b514922d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.527075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a612a-a87c-45c6-b448-b6dda13f05d4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.527893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.528581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.528867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.529143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.529831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.531438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.532308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.536780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.536780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.536780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.536780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.536780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.543117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 913/1802] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:17.551773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.562135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 890/1803] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:17 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:17.564933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.567298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0f4d-0bbd-4dd2-8ee2-4d3130082d69 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.572236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.572969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.573254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.573580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] PATH_INFO: `/v3/projects/65c99460f1564f54a187ce2cfbd70e17/users/3e9a1db6a08045e9b0e664b206bacc0a/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.582077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ca15-4882-4318-a755-470d30a393f1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.589740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.590632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.591000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.591355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] PATH_INFO: `/v3/projects/f88c8ce1b7c14db0b005d1eb500b23a2/users/20a35fbde2774b89bffe7b9e25a31335/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.596833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c99460f1564f54a187ce2cfbd70e17, user_id=3e9a1db6a08045e9b0e664b206bacc0a, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.600815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2e652-88cd-4dda-a06f-1620a6e1dbe5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.610854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88c8ce1b7c14db0b005d1eb500b23a2, user_id=20a35fbde2774b89bffe7b9e25a31335, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.613796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd2748-fe99-4e74-81ef-df6707245e46 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.708355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 891/1804] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:17 2022] PUT /identity/v3/projects/f88c8ce1b7c14db0b005d1eb500b23a2/users/20a35fbde2774b89bffe7b9e25a31335/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:17.710093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 914/1805] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:17 2022] PUT /identity/v3/projects/65c99460f1564f54a187ce2cfbd70e17/users/3e9a1db6a08045e9b0e664b206bacc0a/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:17.720648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.721042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.721220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.722564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.722946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.723155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.812289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] MFA Rules not processed for user `20a35fbde2774b89bffe7b9e25a31335`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:17.813676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:17.823154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] MFA Rules not processed for user `3e9a1db6a08045e9b0e664b206bacc0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:17.826059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:17.890560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 892/1806] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:17.894999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.921500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 915/1807] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:17.923831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:17.968648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eae5ef-a382-42c1-8082-2d045041feed None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:17.973610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:17.974243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:17.974474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:17.974772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:17.975465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:17.978547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c600e-505f-44aa-8889-ccb6d4777126 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:17.996203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b806f-1f58-472a-9fb4-76e3bd3a4bf8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.001944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.001944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.001944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.002586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.002678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.005122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.149355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 916/1808] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/projects => generated 382 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:18.149679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 893/1809] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:17 2022] POST /identity/v3/projects => generated 374 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:18.151029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.151316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.151594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.153442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.171128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:18.172341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:18.226066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1985b0-ad80-4f4f-b893-fefa5c691fd3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.240224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 894/1810] 10.222.0.86 () {58 vars in 1019 bytes} [Tue Jul 26 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:18.240224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.299176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.300081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.300205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.300369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.301636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.303617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.322324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99402c7-f08f-429c-80c8-7d86cfcd5002 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.327034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.327673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.327944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.328219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.328924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.336929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.400775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 917/1811] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:18 2022] POST /identity/v3/users => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:18.404478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.414117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e163-00e4-4b03-a4b8-7c0b72a0c4d7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.418839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.419541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.419829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.420102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.420819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.422412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.432838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 918/1812] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:18.436870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.446907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f29e4e-717b-49bd-9cca-00688fd44f59 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.451565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.452185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.452505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.452780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] PATH_INFO: `/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0/users/6f971bd93a184e0c95a5831b79795cc5/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.467278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7fcfa856f148d0a34661ea0e069aa0, user_id=6f971bd93a184e0c95a5831b79795cc5, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.469928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.478770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 895/1813] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:18 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:18.482541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.493097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e55ed-e5e4-4516-bd81-e977e01ea686 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.497804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.498419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.498749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.500650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.501871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.512056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.518845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 896/1814] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:18.523434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.533278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1da75-e286-4f05-a60a-887b98cd5134 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.539086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.539749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.540028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.540303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] PATH_INFO: `/v3/projects/50e1a2851654426a84018f8db97f65e4/users/ac6ac5d1adf14458917cfee1da79d721/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.556959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e1a2851654426a84018f8db97f65e4, user_id=ac6ac5d1adf14458917cfee1da79d721, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.559792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.574756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 919/1815] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:18 2022] PUT /identity/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0/users/6f971bd93a184e0c95a5831b79795cc5/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:18.579625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.654042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2b16e-3200-40f0-b7a1-ac9ce62e3240 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.659974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.660759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.661133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.661659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.662433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.664520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.672167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 897/1816] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:18 2022] PUT /identity/v3/projects/50e1a2851654426a84018f8db97f65e4/users/ac6ac5d1adf14458917cfee1da79d721/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:18.673301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.675490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:18.678603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:18.686221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 920/1817] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/roles => generated 1427 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:18.689040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.693273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:18.742233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b7425-e93d-4bf5-b22f-9fbc0850a192 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.745957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.746661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.747165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.747661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.755866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.757646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.762383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89093-0afa-464b-a61f-53a572ad4c8f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.766643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 898/1818] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:18.773897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.773897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.774116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.774201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.774201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.776481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.777766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60483e1-03f7-4e44-9499-7d7538cf4173 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.777911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.781708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.782407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.782787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.783175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.786664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 921/1819] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:18.787461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.793009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.794828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.795943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48055-cd3e-48d3-b916-9b2dc4268fc0 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.799917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.800659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.801975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.801975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.809324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.814141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.814743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 899/1820] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:18.814743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.820336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 922/1821] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:18.821850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.925743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56915738-2de1-496b-8726-1801ca13a465 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.931780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.932927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.933230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.933514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] PATH_INFO: `/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0/users/6f971bd93a184e0c95a5831b79795cc5/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.935630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7fcfa856f148d0a34661ea0e069aa0, user_id=6f971bd93a184e0c95a5831b79795cc5, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.938660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539cb2-b47c-4df2-8bf1-f24e26030102 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.940785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c512c0f-b38e-44e2-b211-6c88f26b58ee admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.946463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:18.947103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.947387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.947666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.948370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:18.950849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:18.954931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 923/1822] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:18 2022] PUT /identity/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0/users/6f971bd93a184e0c95a5831b79795cc5/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:18.964312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 900/1823] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:18 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:18.965259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8136d-0124-40e1-b92b-1c8cdbb203d9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:18.965603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8136d-0124-40e1-b92b-1c8cdbb203d9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:18.965886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8136d-0124-40e1-b92b-1c8cdbb203d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:18.969255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:18.986387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce12f-cce4-4e8b-8eba-f7553dbdffd7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:18.997479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:19.003350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f2c8136d-0124-40e1-b92b-1c8cdbb203d9 None None] MFA Rules not processed for user `6f971bd93a184e0c95a5831b79795cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:19.005230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f2c8136d-0124-40e1-b92b-1c8cdbb203d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:19.121912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 924/1824] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:19.147891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:19.148658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:19.149344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:19.149615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] PATH_INFO: `/v3/projects/50e1a2851654426a84018f8db97f65e4/users/ac6ac5d1adf14458917cfee1da79d721/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:19.151886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e1a2851654426a84018f8db97f65e4, user_id=ac6ac5d1adf14458917cfee1da79d721, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:19.154637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:19.176851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 901/1825] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:18 2022] PUT /identity/v3/projects/50e1a2851654426a84018f8db97f65e4/users/ac6ac5d1adf14458917cfee1da79d721/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:19.180482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:19.180482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:19.180482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:19.215019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] MFA Rules not processed for user `ac6ac5d1adf14458917cfee1da79d721`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:19.217201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:19.300641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 925/1826] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:19.411554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:19.417793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:19.431827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:19.433650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:19.506152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24a351-fd1d-4852-bb8d-b66fa8d368c4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:19.511712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:19.512720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:19.513264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:19.513718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:19.536128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcd6ae-f658-4b06-8b3d-2915328ac454 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:19.540145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:19.541224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:19.541625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:19.541976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:19.553958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:19.555729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae988cd-c585-470d-9633-e4d8448c5f33 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:19.568637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 926/1827] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:19.596089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:19.597747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1c060-1acc-471e-9674-0a4d4bfe5cfb service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:19.618387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 902/1828] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:20.484082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:21.404846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eebdfa-c1e9-4628-a9d6-460daa02fd62 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.405811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eebdfa-c1e9-4628-a9d6-460daa02fd62 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.406156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eebdfa-c1e9-4628-a9d6-460daa02fd62 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.409464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 927/1829] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:33:21 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:33:21.421705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.422562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.423012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.453413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:21.454677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:21.517462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac800-fb7f-444f-831d-75479eda2665 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.517748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac800-fb7f-444f-831d-75479eda2665 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.518249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ac800-fb7f-444f-831d-75479eda2665 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.520435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 928/1830] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:33:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:33:21.541418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.542652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.543045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.602989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 903/1831] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:21.619401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:21.621091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:21.635417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 929/1832] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:21.642125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:21.644354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:21.662971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d9bd-cb56-4087-b4a8-e713330f8059 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:21.669544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:21.670191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.670472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.670750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.681440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:21.683120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:21.696466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 904/1833] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:21.827947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:21.833807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:21.863202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218b1cb-25f2-45f2-b076-60d5114c2f0a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:21.887632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:21.888346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:21.888662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:21.888944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:21.908947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:21.911233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:21.959950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 930/1834] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:22.119514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:22.133843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89667-784b-4c8b-bc08-6730c156af6e service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:22.139240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f971bd93a184e0c95a5831b79795cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad7fcfa856f148d0a34661ea0e069aa0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:22.140899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:22.140899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:22.141467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:22.147707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:22.148858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:22.151053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:22.177950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 905/1835] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:22.178855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:22.211791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cc284-4e1f-49dd-9a22-59fd60b1a0a3 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:22.216708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f971bd93a184e0c95a5831b79795cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad7fcfa856f148d0a34661ea0e069aa0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:22.217396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:22.222930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:22.222930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:22.222930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:22.222930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:22.433644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 931/1836] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:22 2022] POST /identity/v3/projects => generated 354 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:22.437420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:22.448070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b684ec4-9d32-4937-b045-efdaa550d813 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:22.522572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f971bd93a184e0c95a5831b79795cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad7fcfa856f148d0a34661ea0e069aa0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:22.523447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:22.523574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:22.523864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:22.524656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:22.524969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:22.526763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:22.540236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 906/1837] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:22.587664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:22.614418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076a736-2628-468a-9d07-87240b89dc29 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:22.617458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f971bd93a184e0c95a5831b79795cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad7fcfa856f148d0a34661ea0e069aa0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:22.618173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:22.619073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:22.619578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] PATH_INFO: `/v3/projects/b88af76ec85e495e96510bb248d63285` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:22.621466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b88af76ec85e495e96510bb248d63285)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:22.623479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:22.834971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88af76ec85e495e96510bb248d63285', 'request_id': 'req-bb077145-7cbb-4ce7-8184-c3749f1790e0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:22.841575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88af76ec85e495e96510bb248d63285', 'request_id': 'req-bb077145-7cbb-4ce7-8184-c3749f1790e0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.037746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88af76ec85e495e96510bb248d63285', 'request_id': 'req-bb077145-7cbb-4ce7-8184-c3749f1790e0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.043529 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:23.065884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] The token cache is being invalidate because project b88af76ec85e495e96510bb248d63285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:23.065884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.065884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb077145-7cbb-4ce7-8184-c3749f1790e0 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.065884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 932/1838] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:22 2022] DELETE /identity/v3/projects/b88af76ec85e495e96510bb248d63285 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:23.119743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:23.129661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919666c9-415e-4df4-9384-61516e0e8978 tempest-VolumeQuotaClassesTest-1816613773 tempest-VolumeQuotaClassesTest-1816613773-project-admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:23.185525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:23.186158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:23.186435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:23.186707 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] PATH_INFO: `/v3/users/20a35fbde2774b89bffe7b9e25a31335` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:23.187873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a35fbde2774b89bffe7b9e25a31335)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:23.189790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:23.526890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a35fbde2774b89bffe7b9e25a31335'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.530917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a35fbde2774b89bffe7b9e25a31335'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.772090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a35fbde2774b89bffe7b9e25a31335'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.772835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a35fbde2774b89bffe7b9e25a31335'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.780746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a35fbde2774b89bffe7b9e25a31335'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:23.798437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 907/1839] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:23 2022] DELETE /identity/v3/users/20a35fbde2774b89bffe7b9e25a31335 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:23.802988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:23.805708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:23.805708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:23.824954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:23.826217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:23.891298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 933/1840] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:23.901728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:23.903223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:23.961358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f66c18-2172-4f6c-8877-dd25dfa2213d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:23.964986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:23.965603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:23.965885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:23.966156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:23.975702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:23.977165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:23.986986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 908/1841] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:24.321687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:24.323523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:24.335308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd38e0-8738-4a48-93e7-b9f5cf6ec879 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:24.340645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:24.341394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:24.341744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:24.342086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] PATH_INFO: `/v3/users/3e9a1db6a08045e9b0e664b206bacc0a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:24.343365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9a1db6a08045e9b0e664b206bacc0a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:24.348543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:24.661189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:24.662641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:24.675323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6c643-8f3f-4368-a711-d99b1c4e89d9 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:24.679930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:24.680714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:24.681087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:24.681256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] PATH_INFO: `/v3/projects/f88c8ce1b7c14db0b005d1eb500b23a2` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:24.682422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88c8ce1b7c14db0b005d1eb500b23a2)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:24.684315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:24.709653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9a1db6a08045e9b0e664b206bacc0a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.710332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9a1db6a08045e9b0e664b206bacc0a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.717329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9a1db6a08045e9b0e664b206bacc0a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.717972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9a1db6a08045e9b0e664b206bacc0a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.729815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df5ea7fa-b5ec-4d8d-8cc0-0472354cc492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9a1db6a08045e9b0e664b206bacc0a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.885891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 934/1842] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:24 2022] DELETE /identity/v3/users/3e9a1db6a08045e9b0e664b206bacc0a => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:24.890563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:24.897505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:24.897505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:24.897675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:24.916487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:24.917678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88c8ce1b7c14db0b005d1eb500b23a2', 'request_id': 'req-e1087276-831b-4c57-8ad3-c75c76f281e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.917812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:24.924224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88c8ce1b7c14db0b005d1eb500b23a2', 'request_id': 'req-e1087276-831b-4c57-8ad3-c75c76f281e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.930346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88c8ce1b7c14db0b005d1eb500b23a2', 'request_id': 'req-e1087276-831b-4c57-8ad3-c75c76f281e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:24.978700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 935/1843] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:24.988984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:24.990360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:25.052024 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ae54-c6bd-4c73-8d82-9a9d991968e3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:25.055747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:25.056358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:25.056674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:25.056945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:25.067774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:25.068946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:25.080176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 936/1844] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:25.110489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] The token cache is being invalidate because project f88c8ce1b7c14db0b005d1eb500b23a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:25.110966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.111884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.114053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 909/1845] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:24 2022] DELETE /identity/v3/projects/f88c8ce1b7c14db0b005d1eb500b23a2 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:25.117925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:25.119604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:25.134926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e6263-57e7-4c38-8e7d-d2cdcfb5fe74 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:25.150913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:25.150913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:25.150913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:25.150913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] PATH_INFO: `/v3/users/ac6ac5d1adf14458917cfee1da79d721` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:25.151754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6ac5d1adf14458917cfee1da79d721)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:25.151754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:25.634192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6ac5d1adf14458917cfee1da79d721'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.635194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6ac5d1adf14458917cfee1da79d721'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.653229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6ac5d1adf14458917cfee1da79d721'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.654649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6ac5d1adf14458917cfee1da79d721'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.675405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6ac5d1adf14458917cfee1da79d721'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:25.857059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 937/1846] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:25 2022] DELETE /identity/v3/users/ac6ac5d1adf14458917cfee1da79d721 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:26.831485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:26.833799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:26.848499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087276-831b-4c57-8ad3-c75c76f281e4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:26.920781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:26.921643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:26.921964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:26.922347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] PATH_INFO: `/v3/projects/65c99460f1564f54a187ce2cfbd70e17` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:26.923533 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c99460f1564f54a187ce2cfbd70e17)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:26.925466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:27.147671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c99460f1564f54a187ce2cfbd70e17', 'request_id': 'req-31e52061-abc9-4563-9bec-e2d48f0172e7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.156620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c99460f1564f54a187ce2cfbd70e17', 'request_id': 'req-31e52061-abc9-4563-9bec-e2d48f0172e7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.160095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c99460f1564f54a187ce2cfbd70e17', 'request_id': 'req-31e52061-abc9-4563-9bec-e2d48f0172e7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.430468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] The token cache is being invalidate because project 65c99460f1564f54a187ce2cfbd70e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:27.430468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.430468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.432604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:27.433095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 910/1847] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:26 2022] DELETE /identity/v3/projects/65c99460f1564f54a187ce2cfbd70e17 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:27.435011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:27.437292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:27.438752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:27.463349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443a5e1-eaae-48f3-9a49-c0b747a6c76f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:27.498050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:27.517673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52061-abc9-4563-9bec-e2d48f0172e7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:27.522250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:27.522889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:27.523170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:27.523443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] PATH_INFO: `/v3/users/6f971bd93a184e0c95a5831b79795cc5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:27.524635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f971bd93a184e0c95a5831b79795cc5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:27.526942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:27.548226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:27.549433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:27.550165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:27.550165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] PATH_INFO: `/v3/projects/50e1a2851654426a84018f8db97f65e4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:27.552922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e1a2851654426a84018f8db97f65e4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:27.555141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:27.796117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f971bd93a184e0c95a5831b79795cc5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.796280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e1a2851654426a84018f8db97f65e4', 'request_id': 'req-574980a8-8d25-49a0-87a0-fa4682ee3e0d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.796723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f971bd93a184e0c95a5831b79795cc5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:27.804323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e1a2851654426a84018f8db97f65e4', 'request_id': 'req-574980a8-8d25-49a0-87a0-fa4682ee3e0d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.003225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f971bd93a184e0c95a5831b79795cc5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.003851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f971bd93a184e0c95a5831b79795cc5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.012449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-19402f3d-4179-4221-bdc5-e6bea33f1dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f971bd93a184e0c95a5831b79795cc5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.030659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 911/1848] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:27 2022] DELETE /identity/v3/users/6f971bd93a184e0c95a5831b79795cc5 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:28.135533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e1a2851654426a84018f8db97f65e4', 'request_id': 'req-574980a8-8d25-49a0-87a0-fa4682ee3e0d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.147891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] The token cache is being invalidate because project 50e1a2851654426a84018f8db97f65e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:28.148180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.152163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-574980a8-8d25-49a0-87a0-fa4682ee3e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.152163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 938/1849] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:27 2022] DELETE /identity/v3/projects/50e1a2851654426a84018f8db97f65e4 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:28.155045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.155339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.155619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.177510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:28.178791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:28.238713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 912/1850] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:28.247518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.247808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.248081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.268538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:28.270272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:28.278501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 939/1851] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:28.283757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:28.297180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734cd68-bd49-466a-90dc-eccf32fe33b9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:28.302311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:28.303260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.303550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.303823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.304946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:28.305406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:28.307006 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:28.321202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 913/1852] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:28.329157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:28.339173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdd53c-1449-4532-a57c-f9bae23b837a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:28.344151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:28.344787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.345068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.345343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.345998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:28.348667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:28.527640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 940/1853] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:28 2022] POST /identity/v3/projects => generated 368 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:28.530858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:28.535202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:28.537234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:28.557955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848b9aa-11ab-4918-847b-f9d7fb3a5768 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:28.664950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:28.665933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.666415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.666887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] PATH_INFO: `/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.668437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad7fcfa856f148d0a34661ea0e069aa0)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:28.670845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:28.683554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a161c1-8134-43ce-8c40-896725885f6d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:28.697864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:28.698576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:28.699305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:28.699584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:28.702421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:28.704414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:28.916105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:28.925580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 941/1854] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:28 2022] POST /identity/v3/users => generated 285 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:28.932834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:28.933906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad7fcfa856f148d0a34661ea0e069aa0', 'request_id': 'req-48413172-5ab7-4a71-b8ab-1d390d0012c6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.941619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad7fcfa856f148d0a34661ea0e069aa0', 'request_id': 'req-48413172-5ab7-4a71-b8ab-1d390d0012c6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.949833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad7fcfa856f148d0a34661ea0e069aa0', 'request_id': 'req-48413172-5ab7-4a71-b8ab-1d390d0012c6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:28.958797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423002ad-c65c-465f-8193-fdbc8d577c4f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.014679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.015549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.015831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.016101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.016838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.018422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.028423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 942/1855] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:28 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:29.032223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.042047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb8ec3-16e9-4f55-bfe9-b3005add4a23 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.046543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.047100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.047489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.047661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.048308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.049857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.059775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 943/1856] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:29 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:29.063601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.072801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455edb1d-09a0-4bc2-bb1a-823274001268 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.077584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.105290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.105290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.105290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] PATH_INFO: `/v3/projects/ea45219d9c534d8b9c03ea2e08c14e84/users/b6a8b0f9d1784055a23df31501a230d3/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.105290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea45219d9c534d8b9c03ea2e08c14e84, user_id=b6a8b0f9d1784055a23df31501a230d3, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.105290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c4ba0-6162-4c9c-aaf5-d046042b5e3c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.294075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] The token cache is being invalidate because project ad7fcfa856f148d0a34661ea0e069aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:29.294577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:29.295005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-48413172-5ab7-4a71-b8ab-1d390d0012c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:29.296569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 914/1857] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:28 2022] DELETE /identity/v3/projects/ad7fcfa856f148d0a34661ea0e069aa0 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:29.303543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0a16-241d-4c44-ba39-3620979ac14c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.303834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0a16-241d-4c44-ba39-3620979ac14c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.304113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0a16-241d-4c44-ba39-3620979ac14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.324847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e34e0a16-241d-4c44-ba39-3620979ac14c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:29.326060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e34e0a16-241d-4c44-ba39-3620979ac14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:29.334578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 915/1858] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:29.340657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 944/1859] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:29 2022] PUT /identity/v3/projects/ea45219d9c534d8b9c03ea2e08c14e84/users/b6a8b0f9d1784055a23df31501a230d3/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:29.343722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.344014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.344296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.355228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.355520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.355802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.363885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:29.365297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:29.383168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] MFA Rules not processed for user `b6a8b0f9d1784055a23df31501a230d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:29.384428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:29.425402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 916/1860] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:29.429734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.438420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4698d3-0704-497d-8592-7538600dedb9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.443518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.444111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.444419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.444692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.445380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:29.445683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.447295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.449561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 945/1861] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:29.454211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.455757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:29.459650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 917/1862] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:33:29.463813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.473519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a057ef-be32-44f2-8b9c-cfdf20f6284a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.477117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ab67b-95e6-4b12-8ccf-907db7dc0b66 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.479485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.480180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.488248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.488248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.488248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.489042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.489042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.489042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.489042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.489314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.489384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.489384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.693747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 918/1863] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/projects => generated 392 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:29.700675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.710325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 946/1864] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/projects => generated 368 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:29.712712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:29.814081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61dcaf-2327-4e8f-827f-72ff16090f2f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.820330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135ca6-b7a5-4e2a-98b8-54b921093f2c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:29.820495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.821467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.821854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.822228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.823063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.825215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:29.825715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:29.826344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:29.826630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:29.826903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:29.827565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:29.829509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.023786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 919/1865] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:30.027871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.031543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 947/1866] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:29 2022] POST /identity/v3/users => generated 291 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:30.033544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.045080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645f4cf-7552-46ec-b678-5f87ab1c6c82 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.047700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b60003-4d31-49ab-af3f-f6f6dc7fd282 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.052749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.052749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.052749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.052749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.053767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.055477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.057002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.057638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.057923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.058196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.058933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.061330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.077190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 948/1867] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:30 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:30.077190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.083862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f011c-be84-4c52-930e-1d9e6a16b3b0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.088796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.089207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 920/1868] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:30 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:30.089467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.089747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.090020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] PATH_INFO: `/v3/projects/f3f507c6098c44c5840f5bec0b000d41/users/abd7566be25f4f7aa0221e535a6c4828/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.093301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.103322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c764c4-da26-4a5a-ad53-4865ddf810f1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.107327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f507c6098c44c5840f5bec0b000d41, user_id=abd7566be25f4f7aa0221e535a6c4828, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.108149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.108803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.109089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.109365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.110060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.110749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.111636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.123053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 921/1869] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:30 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:30.127351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.138173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac176cef-a260-40db-8171-115c0c91895b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.143310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.144050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.144411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.145319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] PATH_INFO: `/v3/projects/ad9789ab29684338b7804851988795af/users/06b941a0751f40258118104b19d97008/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.159900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9789ab29684338b7804851988795af, user_id=06b941a0751f40258118104b19d97008, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.163325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7025f-230a-4831-9c9f-67d72d625c91 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.260465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 949/1870] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:30 2022] PUT /identity/v3/projects/f3f507c6098c44c5840f5bec0b000d41/users/abd7566be25f4f7aa0221e535a6c4828/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:30.264210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.299203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 922/1871] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:30 2022] PUT /identity/v3/projects/ad9789ab29684338b7804851988795af/users/06b941a0751f40258118104b19d97008/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:30.308330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.310314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.310314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.329898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab31d3-6da5-421d-8686-bd990e6e9412 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.338493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] MFA Rules not processed for user `06b941a0751f40258118104b19d97008`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:30.339787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:30.393765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.394491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.395472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.395852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.396692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.398700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.413274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 923/1872] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:30.415758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 950/1873] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:30 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:30.422271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.422473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.434729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe857cd-1871-410e-b5df-3e089250ba94 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.439775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.440786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.441131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.441465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.442737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.444370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f239-4c38-4533-9543-734086b8e612 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.444798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.449166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.449921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.450277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.450656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] PATH_INFO: `/v3/projects/f3f507c6098c44c5840f5bec0b000d41/users/abd7566be25f4f7aa0221e535a6c4828/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.452781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f507c6098c44c5840f5bec0b000d41, user_id=abd7566be25f4f7aa0221e535a6c4828, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.455775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1c05-7c42-4128-99c0-1cd7d82045a3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.466980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 924/1874] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:30 2022] PUT /identity/v3/projects/f3f507c6098c44c5840f5bec0b000d41/users/abd7566be25f4f7aa0221e535a6c4828/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:30.478538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.484696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.484696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.508034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] MFA Rules not processed for user `abd7566be25f4f7aa0221e535a6c4828`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:30.509278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:30.584539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 925/1875] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:30.589746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 951/1876] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:30 2022] POST /identity/v3/projects => generated 392 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:30.593654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.596660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:30.598917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:30.656936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043c89a-2032-4848-b557-f1039bf9ffe0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.658452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdaed5-c552-4baf-bd56-b500db496232 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:30.661088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.661734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.662009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.662277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.716796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:30.717419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:30.717706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:30.717984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:30.718652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.720607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.721608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:30.723567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:30.733104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 952/1877] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:31.078794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 926/1878] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:30 2022] POST /identity/v3/users => generated 303 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:31.082994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.100587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ea274-edc3-4354-82e8-2c1a46a55e56 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.106069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.106686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.106964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.107235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.107948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.110000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.110407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.121439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 953/1879] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:31 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:31.123395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e98b-3a10-434a-ba8b-88e7e47061ca admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.137593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.137593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bcd0-7ec3-4ed8-8bae-f6993146dcaa admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.142355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.142854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.143131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.143480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.143762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.143872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.144176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.144556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] PATH_INFO: `/v3/projects/685378a711eb476bbf77a99253dafb01/users/030b8daa20e2498dbe4b3eb9dc13c85d/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.166851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=685378a711eb476bbf77a99253dafb01, user_id=030b8daa20e2498dbe4b3eb9dc13c85d, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.169912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.220636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.222427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.239450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 927/1880] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:31.364857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 954/1881] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:31 2022] PUT /identity/v3/projects/685378a711eb476bbf77a99253dafb01/users/030b8daa20e2498dbe4b3eb9dc13c85d/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:31.368727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.379898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d035-7be2-4032-8fdd-709e8b29f95a service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.443081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.443776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.444058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.444332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.445180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.446714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db381a9-42d5-4798-bcb2-2b2e9598819a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.456481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 928/1882] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:31 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:31.460690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.470855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a9073-65ca-4303-a3b2-ef2aaffe2bba admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.476050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.481708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.481708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.481708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] PATH_INFO: `/v3/projects/685378a711eb476bbf77a99253dafb01/users/030b8daa20e2498dbe4b3eb9dc13c85d/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.481708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=685378a711eb476bbf77a99253dafb01, user_id=030b8daa20e2498dbe4b3eb9dc13c85d, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.482478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.492953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 955/1883] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:31 2022] PUT /identity/v3/projects/685378a711eb476bbf77a99253dafb01/users/030b8daa20e2498dbe4b3eb9dc13c85d/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:31.503077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.503429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.503694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.533302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] MFA Rules not processed for user `030b8daa20e2498dbe4b3eb9dc13c85d`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:31.534355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:31.608324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 929/1884] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:31.611587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.613837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:31.693241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d841c3-d750-4348-aac0-43cbf213dfbc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.699954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.700159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.700716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.700841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] PATH_INFO: `/v3/users/62c98d94542947a4842d4ad9c5123153` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.702058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c98d94542947a4842d4ad9c5123153)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.704046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.749345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:31.751097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:31.808978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f103f-64e7-4184-bf6f-9e98caeb7e54 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:31.812733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:31.813345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:31.813634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:31.813935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:31.823567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:31.825101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacb404-a245-42f5-8c81-9eeb0621a23e service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:31.834575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 930/1885] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:31.889533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:31.954854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c98d94542947a4842d4ad9c5123153'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:31.955476 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c98d94542947a4842d4ad9c5123153'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:31.962371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c98d94542947a4842d4ad9c5123153'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:31.963005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c98d94542947a4842d4ad9c5123153'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:31.974539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c98d94542947a4842d4ad9c5123153'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:32.118369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 956/1886] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:31 2022] DELETE /identity/v3/users/62c98d94542947a4842d4ad9c5123153 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:32.123042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:32.123329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:32.123625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:32.143415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:32.145371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:32.189234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:32.190652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:32.235928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 931/1887] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:32.242812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:32.244290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:32.263686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26fe24-d419-4886-a735-62f410021364 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:32.267952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:32.268606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:32.268893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:32.269211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:32.271560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:32.321532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f98ec-4fcd-49fa-b595-34f72ed587fa None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:32.326508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:32.326656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:32.326979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:32.327271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:32.335777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:32.337418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:32.337613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:32.339075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:32.349660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 932/1888] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:32.358966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 957/1889] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:33.068918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:33.070379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:33.084286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8036-adea-4fad-8d91-3fa9fe333559 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:33.090766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:33.091178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:33.091455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:33.091726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] PATH_INFO: `/v3/projects/ce9690eefda448588e15513aae708a58` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:33.094082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9690eefda448588e15513aae708a58)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:33.096002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:33.210721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9690eefda448588e15513aae708a58', 'request_id': 'req-7f2984ef-3364-4017-b2cd-23768c3535f8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.216925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9690eefda448588e15513aae708a58', 'request_id': 'req-7f2984ef-3364-4017-b2cd-23768c3535f8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.222668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9690eefda448588e15513aae708a58', 'request_id': 'req-7f2984ef-3364-4017-b2cd-23768c3535f8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.377531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] The token cache is being invalidate because project ce9690eefda448588e15513aae708a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:33.377867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.378775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7f2984ef-3364-4017-b2cd-23768c3535f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.381008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 933/1890] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:33 2022] DELETE /identity/v3/projects/ce9690eefda448588e15513aae708a58 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:33.385063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:33.397245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278dde5a-7034-43fd-9dcf-e9795b5b8304 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:33.453648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:33.454283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:33.454567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:33.454842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] PATH_INFO: `/v3/users/82b68d1ae63b4723a059868ff61a84b3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:33.455991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b68d1ae63b4723a059868ff61a84b3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:33.457979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:33.621795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:33.691818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b68d1ae63b4723a059868ff61a84b3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.693205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b68d1ae63b4723a059868ff61a84b3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.701437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b68d1ae63b4723a059868ff61a84b3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.710390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b68d1ae63b4723a059868ff61a84b3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.727252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02035199-cb65-4362-adb2-6e183e4ad7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b68d1ae63b4723a059868ff61a84b3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:33.861313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 958/1891] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:33 2022] DELETE /identity/v3/users/82b68d1ae63b4723a059868ff61a84b3 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:34.308757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.309293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.309293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:34.310656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 934/1892] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:33:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:33:34.318014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.318518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.318867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:34.342660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:34.343804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:34.344928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:34.361424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:34.425953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de656bb-c38e-45b2-9533-2dfb8373a63e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:34.430508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:34.431151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.431434 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.431709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] PATH_INFO: `/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:34.432920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4146f62417ba416b9f4d730bcc4e9ff0)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:34.433920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 959/1893] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:34.434870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:34.449254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:34.450739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:34.466234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6c4f2-7a19-471f-b4e9-c3ee4da774ec None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:34.472409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:34.473054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.473335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.473605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:34.483114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:34.485945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ac7a7-2664-479a-9547-84c063a11fee service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:34.499104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 960/1894] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:34.604184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4146f62417ba416b9f4d730bcc4e9ff0', 'request_id': 'req-b14c7308-59c1-4264-96d3-b1498e43d138'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:34.611469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4146f62417ba416b9f4d730bcc4e9ff0', 'request_id': 'req-b14c7308-59c1-4264-96d3-b1498e43d138'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:34.619399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4146f62417ba416b9f4d730bcc4e9ff0', 'request_id': 'req-b14c7308-59c1-4264-96d3-b1498e43d138'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:34.830876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] The token cache is being invalidate because project 4146f62417ba416b9f4d730bcc4e9ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:34.830876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:34.830876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b14c7308-59c1-4264-96d3-b1498e43d138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:34.830876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 935/1895] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:34 2022] DELETE /identity/v3/projects/4146f62417ba416b9f4d730bcc4e9ff0 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:34.831276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.831276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.831442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:34.861335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:34.862585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:34.936906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 961/1896] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:34.952641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:34.953162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:34.953585 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.009926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:35.011959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:35.025742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 936/1897] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:35.030594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.048706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120631c-d97a-4b9c-a29a-5c27d64e5676 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.061829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.062665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.063952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.065721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.068280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:35.068941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.072589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.093979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.107216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 962/1898] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:35.118948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dd684-5aea-4261-8e57-59ecffbeccfb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.124889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.125525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.125814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.126089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.126911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.129491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.287608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.300168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 937/1899] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:35 2022] POST /identity/v3/projects => generated 362 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:35.306219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301724-2e0c-47b2-8cfb-e0d0ef98d961 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.400200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.400864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.401149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.401427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.402099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.404038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.596831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 963/1900] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:35 2022] POST /identity/v3/users => generated 282 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:35.598020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.608494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd6101-9fa4-4606-abab-559718c1e353 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.613055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.614268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.614548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.614824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.615523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.617216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.628766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 938/1901] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:35 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:35.633161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.648449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2aa8-e652-4fde-ada5-2fa292d296f0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.662453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.663240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.663532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.663841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.664686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.666302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fc098-a93e-45e7-8430-ad0a92dc3c22 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.678409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 964/1902] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:35 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:35.682962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.693486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8d91-5f23-471b-9766-2b4a18071e05 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:35.698203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:35.698879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.699221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.699684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] PATH_INFO: `/v3/projects/1338af236bd741399bcfd571de54ca12/users/8a1ad1f349c14ea6bc8797b5e45bdf56/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.715384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1338af236bd741399bcfd571de54ca12, user_id=8a1ad1f349c14ea6bc8797b5e45bdf56, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:35.718326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:35.815311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 939/1903] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:35 2022] PUT /identity/v3/projects/1338af236bd741399bcfd571de54ca12/users/8a1ad1f349c14ea6bc8797b5e45bdf56/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:35.825877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:35.826187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:35.826441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:35.857212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] MFA Rules not processed for user `8a1ad1f349c14ea6bc8797b5e45bdf56`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:35.858457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:35.873744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:35.928541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 965/1904] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:35.963383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:35.965128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:36.022921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5098a-ea7c-4b16-9ca1-f80a3cac0291 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:36.026583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:36.027204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:36.027472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:36.027745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:36.039424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:36.040928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d678d-6b5c-4376-b254-0318c357a712 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:36.052705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 940/1905] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:36.068192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:36.139273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc747add-f786-4901-8aa7-45f8298ea75a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:36.144424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:36.146154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:36.146154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:36.146154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] PATH_INFO: `/v3/users/8a1ad1f349c14ea6bc8797b5e45bdf56` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:36.146909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1ad1f349c14ea6bc8797b5e45bdf56)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:36.148981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:36.424184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1ad1f349c14ea6bc8797b5e45bdf56'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:36.425105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1ad1f349c14ea6bc8797b5e45bdf56'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:36.432853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1ad1f349c14ea6bc8797b5e45bdf56'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:36.433631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1ad1f349c14ea6bc8797b5e45bdf56'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:36.447593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1ad1f349c14ea6bc8797b5e45bdf56'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:36.571152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 966/1906] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:36 2022] DELETE /identity/v3/users/8a1ad1f349c14ea6bc8797b5e45bdf56 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:36.575754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:36.576054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:36.576741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:36.596764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:36.598515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:36.686758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 941/1907] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:36.696099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:36.697645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:36.758581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961e591-eb3a-4b27-b979-0896891387f0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:36.762418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:36.763126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:36.763479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:36.763852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:36.773538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:36.775101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ffbf7-d0c6-415c-a429-aa0a62d23aef service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:36.784993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 967/1908] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:37.369860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:37.371856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:37.404711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b268d2-0961-4c0a-895e-92248e01e1a6 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:37.420761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:37.421430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:37.421712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:37.421987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] PATH_INFO: `/v3/projects/1338af236bd741399bcfd571de54ca12` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:37.423885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1338af236bd741399bcfd571de54ca12)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:37.426009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:37.655997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:37.674500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1338af236bd741399bcfd571de54ca12', 'request_id': 'req-94a27940-7918-453d-87e3-fc5c9d642c6e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:37.682098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1338af236bd741399bcfd571de54ca12', 'request_id': 'req-94a27940-7918-453d-87e3-fc5c9d642c6e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:37.691986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1338af236bd741399bcfd571de54ca12', 'request_id': 'req-94a27940-7918-453d-87e3-fc5c9d642c6e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:37.792915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] The token cache is being invalidate because project 1338af236bd741399bcfd571de54ca12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:37.793302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:37.794241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94a27940-7918-453d-87e3-fc5c9d642c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:37.796398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 942/1909] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:37 2022] DELETE /identity/v3/projects/1338af236bd741399bcfd571de54ca12 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:37.801518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:37.803078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:37.803560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:37.826614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:37.827981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:37.890869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 968/1910] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:37.900222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:37.900542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:37.900823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:37.921710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:37.923307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:37.931631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 943/1911] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:37.935908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:37.945439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2368b6-4ab0-42a7-ba38-b0ceaebee926 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:37.950492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:37.951098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:37.951378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:37.951648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:37.952330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:37.952666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:37.954254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:37.966074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 969/1912] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:37.972239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:37.982398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b56951-b671-4ae1-8db0-1c1f4267fbcb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:37.987530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:37.988214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:37.988616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:37.988948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:37.989699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:37.991639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.134495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 944/1913] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:37 2022] POST /identity/v3/projects => generated 376 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:38.139047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.157540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7cf28-d124-4d0c-8037-1ececc02017c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.229441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.230055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.230331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.230600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.231254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.233182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.313469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.315195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:38.328957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903058e-aea5-449e-a7b1-18742f78a390 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.337968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.341337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 970/1914] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:38 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:38.342247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.342704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.343154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] PATH_INFO: `/v3/users/b6a8b0f9d1784055a23df31501a230d3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.344708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a8b0f9d1784055a23df31501a230d3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.346269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.354969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.361817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac40391-1ae6-42fd-a3fd-cdea53b760a1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.375626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.376510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.376963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.377382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.378236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.383567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.412551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 971/1915] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:38 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:38.419877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.430480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cdde7-0a18-435a-8aed-644df2e841ab admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.435597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.440701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.441104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.441504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.445749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.445749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.456336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 972/1916] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:38 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:38.460694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.485309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49514e-c097-4f1f-a05b-1dd302976b88 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.485309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.485309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.485309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.485702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] PATH_INFO: `/v3/projects/5d2cb91725814907950373291fdfa528/users/a92fb0ef6047443da1de4de87f092ea9/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.501608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2cb91725814907950373291fdfa528, user_id=a92fb0ef6047443da1de4de87f092ea9, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.504436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73afa7-9e7f-4133-8b94-8c649cf33822 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.565979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a8b0f9d1784055a23df31501a230d3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:38.567092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a8b0f9d1784055a23df31501a230d3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:38.611128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 973/1917] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:38 2022] PUT /identity/v3/projects/5d2cb91725814907950373291fdfa528/users/a92fb0ef6047443da1de4de87f092ea9/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:38.624694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.625163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.625594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.649977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a8b0f9d1784055a23df31501a230d3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:38.651007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a8b0f9d1784055a23df31501a230d3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:38.664576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] MFA Rules not processed for user `a92fb0ef6047443da1de4de87f092ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:38.666145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:38.667048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b52c696a-1cff-4494-b5df-097fa5c0ba35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a8b0f9d1784055a23df31501a230d3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:38.697625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 945/1918] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:38 2022] DELETE /identity/v3/users/b6a8b0f9d1784055a23df31501a230d3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:38.699621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.699921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.700352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.731533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:38.737609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:38.740606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:38.762839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 974/1919] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:38.773590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.775006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:38.814581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 946/1920] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:38.825023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:38.826442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:38.853458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2e2cc-02d6-489e-907e-e5a04a777ae7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.858258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.858887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.859170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.859451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.860121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.862551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.886574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a583e53-88cc-4025-85de-10d499758ab5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:38.890885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:38.891708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:38.892022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:38.892301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:38.903044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:38.904635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:38.917751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 947/1921] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:38.997171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 975/1922] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:38 2022] POST /identity/v3/projects => generated 378 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:38.998470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.008230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4b92-2ed3-4a47-ab94-873b710bf125 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.074619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.074619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.074619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.074619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.074938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.074938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.080512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.082311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.095595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0c0fd-3ac8-4be0-8a1b-0815ba9f1327 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.100804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.101437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.101720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.101998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] PATH_INFO: `/v3/users/06b941a0751f40258118104b19d97008` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.103316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b941a0751f40258118104b19d97008)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.105187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.164417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 948/1923] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:38 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:39.168308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.179037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f4373-d509-4358-85d8-b27aafc82f5e admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.183939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.184600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.184882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.185157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.185858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.187468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.197805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 949/1924] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:39 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:39.204889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.217769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a2f78-4274-496a-8974-991dba2173db admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.224268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.225116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.229705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.229705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.229705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.229705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.245697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 950/1925] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:39 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:39.247134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.260183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64acb72-50dd-4145-9545-de135f36adcc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.265641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.266425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.266743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.267042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] PATH_INFO: `/v3/projects/28457df82331453dbf9491cc3c079351/users/52654794fb45493b841676476bc4f494/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.286257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28457df82331453dbf9491cc3c079351, user_id=52654794fb45493b841676476bc4f494, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.289042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e17f9-321a-42d5-86d1-f564f4dbe43f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.314290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b941a0751f40258118104b19d97008'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.314950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b941a0751f40258118104b19d97008'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.323346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b941a0751f40258118104b19d97008'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.324079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b941a0751f40258118104b19d97008'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.336627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-afc9c76b-1736-41d5-8a80-a599d45eee8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b941a0751f40258118104b19d97008'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.370695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 951/1926] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:39 2022] PUT /identity/v3/projects/28457df82331453dbf9491cc3c079351/users/52654794fb45493b841676476bc4f494/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:39.381949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.382297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.382576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.411829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] MFA Rules not processed for user `52654794fb45493b841676476bc4f494`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:39.413216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.414810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 976/1927] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:39 2022] DELETE /identity/v3/users/06b941a0751f40258118104b19d97008 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:39.419066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.419396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.419643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.439378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:39.440693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.482954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 952/1928] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:39.493321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.494839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.514580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 977/1929] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:39.521461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.527583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.549465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4547-0a60-4c34-9fe6-d0dac7770dd2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.554741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.555390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.555670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.555942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] PATH_INFO: `/v3/projects/ea45219d9c534d8b9c03ea2e08c14e84` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.557116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea45219d9c534d8b9c03ea2e08c14e84)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.559996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.581772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3538d43-ccc7-4298-9def-7031958e6f50 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.587184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.587785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.588065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.588395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.589098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.591291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.673312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 953/1930] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:39 2022] POST /identity/v3/projects => generated 378 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:39.674149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.678360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.685524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea45219d9c534d8b9c03ea2e08c14e84', 'request_id': 'req-56741353-9f8f-4a1d-8870-542eaa60fc6c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.691671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea45219d9c534d8b9c03ea2e08c14e84', 'request_id': 'req-56741353-9f8f-4a1d-8870-542eaa60fc6c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.733098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd507a7-727a-496c-bdb6-3b42f924d0b7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.736587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.737192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.737470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.737741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.804127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.806186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.817831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 954/1931] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:39.818489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.826555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea45219d9c534d8b9c03ea2e08c14e84', 'request_id': 'req-56741353-9f8f-4a1d-8870-542eaa60fc6c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.829314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936cd7-200e-4dc7-9ce6-c51bc3355b0d service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.836427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.837075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.837419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.837701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.838314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.840237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:39.859107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] The token cache is being invalidate because project ea45219d9c534d8b9c03ea2e08c14e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:39.859639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.861115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:39.869168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 978/1932] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:39 2022] DELETE /identity/v3/projects/ea45219d9c534d8b9c03ea2e08c14e84 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:39.873302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:39.874830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:39.981725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741353-9f8f-4a1d-8870-542eaa60fc6c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:39.987406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:39.988081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:39.989435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:39.989435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] PATH_INFO: `/v3/users/abd7566be25f4f7aa0221e535a6c4828` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:39.991750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7566be25f4f7aa0221e535a6c4828)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:39.993982 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.023732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 955/1933] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:39 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:40.028269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.032685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.053713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db420f-768d-48a9-8865-127ce3c34059 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.059556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.060713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.061179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.061974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.062919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.064700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.081229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 956/1934] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:40 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:40.093114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.103668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd99c3a-a980-43fc-935a-14dbd841fc08 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.109203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.109893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.110173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.115250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] PATH_INFO: `/v3/projects/b09ad34296d0497ab7d3a4842e174fd0/users/1c3e248a4e2d4a5bbd3965fab30208f0/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.127819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09ad34296d0497ab7d3a4842e174fd0, user_id=1c3e248a4e2d4a5bbd3965fab30208f0, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.131076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.222602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 957/1935] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:40 2022] PUT /identity/v3/projects/b09ad34296d0497ab7d3a4842e174fd0/users/1c3e248a4e2d4a5bbd3965fab30208f0/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:40.226398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.233458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7566be25f4f7aa0221e535a6c4828'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.234880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7566be25f4f7aa0221e535a6c4828'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.242044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7566be25f4f7aa0221e535a6c4828'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.242796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7566be25f4f7aa0221e535a6c4828'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.256192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7566be25f4f7aa0221e535a6c4828'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.309335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ae463-9c19-45b8-8047-5c52e776d410 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.314082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.314698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.314980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.315258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.315946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.318102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.328570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 958/1936] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:40 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:40.329721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 979/1937] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:39 2022] DELETE /identity/v3/users/abd7566be25f4f7aa0221e535a6c4828 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:40.332832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.334489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.347315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ed66d-b70b-40ca-a060-b3ab3f6ed6bc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.403847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.404547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.404831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.405105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] PATH_INFO: `/v3/projects/b09ad34296d0497ab7d3a4842e174fd0/users/1c3e248a4e2d4a5bbd3965fab30208f0/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.407049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09ad34296d0497ab7d3a4842e174fd0, user_id=1c3e248a4e2d4a5bbd3965fab30208f0, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.409788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560d875-0fdf-4271-ac5e-b2a3e380428c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.417731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.419224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.435132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 959/1938] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:40 2022] PUT /identity/v3/projects/b09ad34296d0497ab7d3a4842e174fd0/users/1c3e248a4e2d4a5bbd3965fab30208f0/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:40.436693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.436995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.437277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.438373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b6fc5-98c4-4e0b-b115-bf6c0cf59418 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.466265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:40.474038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] MFA Rules not processed for user `1c3e248a4e2d4a5bbd3965fab30208f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:40.478320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.520631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.521963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.522393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.522838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] PATH_INFO: `/v3/projects/ad9789ab29684338b7804851988795af` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.524981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9789ab29684338b7804851988795af)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.528183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.560800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:40.634629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 960/1939] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:40.640788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.643076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.710376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dec64-c190-4b11-9dcf-00f3e0e0c000 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.713260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9789ab29684338b7804851988795af', 'request_id': 'req-73533d56-a992-41db-817f-60d27bfcb8e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.729028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9789ab29684338b7804851988795af', 'request_id': 'req-73533d56-a992-41db-817f-60d27bfcb8e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.783174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.784000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.784424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.784870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.788650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:40.872518 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9789ab29684338b7804851988795af', 'request_id': 'req-73533d56-a992-41db-817f-60d27bfcb8e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.881637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.883272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:40.895350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 961/1940] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:40.898802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] The token cache is being invalidate because project ad9789ab29684338b7804851988795af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:40.899280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.900702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:40.904573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 980/1941] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:40 2022] DELETE /identity/v3/projects/ad9789ab29684338b7804851988795af => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:40.908804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:40.982923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f4f78-581b-4d9f-b6cc-a2a0cc542388 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:40.989125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:40.989857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:40.990359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:40.990642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] PATH_INFO: `/v3/users/030b8daa20e2498dbe4b3eb9dc13c85d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:40.992038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] RBAC: Authorizing `identity:delete_user(user_id=030b8daa20e2498dbe4b3eb9dc13c85d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:40.995515 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:41.141218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:41.143271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:41.159871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73533d56-a992-41db-817f-60d27bfcb8e8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:41.165924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:41.168837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.169046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.169323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] PATH_INFO: `/v3/projects/f3f507c6098c44c5840f5bec0b000d41` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.170556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f507c6098c44c5840f5bec0b000d41)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:41.172513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:41.291091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030b8daa20e2498dbe4b3eb9dc13c85d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.291091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030b8daa20e2498dbe4b3eb9dc13c85d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.316433 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f507c6098c44c5840f5bec0b000d41', 'request_id': 'req-cc74071d-6241-40f9-80aa-170764f5f772'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.324622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f507c6098c44c5840f5bec0b000d41', 'request_id': 'req-cc74071d-6241-40f9-80aa-170764f5f772'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.381199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030b8daa20e2498dbe4b3eb9dc13c85d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.382226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030b8daa20e2498dbe4b3eb9dc13c85d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.392765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-6ad90cde-9ee0-4a09-b05e-72e66070654c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030b8daa20e2498dbe4b3eb9dc13c85d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.423952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 962/1942] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:40 2022] DELETE /identity/v3/users/030b8daa20e2498dbe4b3eb9dc13c85d => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:41.449089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f507c6098c44c5840f5bec0b000d41', 'request_id': 'req-cc74071d-6241-40f9-80aa-170764f5f772'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.467197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] The token cache is being invalidate because project f3f507c6098c44c5840f5bec0b000d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:41.467197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.467197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cc74071d-6241-40f9-80aa-170764f5f772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:41.467538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 981/1943] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:41 2022] DELETE /identity/v3/projects/f3f507c6098c44c5840f5bec0b000d41 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:41.474961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.475324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.475730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.501521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:41.503142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:41.578756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 963/1944] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:41.586782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.587074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.588134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.611747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:41.613004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:41.621638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 982/1945] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:41.626120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:41.636514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230771fa-d50b-4f36-9814-58607fb5c117 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:41.641350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:41.642010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.642293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.642579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.643274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:41.643579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:41.645252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:41.661551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 964/1946] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:41.667816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:41.678703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f8c6b-14a1-4ca4-8793-607b96f8a4b2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:41.683760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:41.684379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.684697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.684968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.685636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:41.688981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:41.804912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 983/1947] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:41 2022] POST /identity/v3/projects => generated 380 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:41.809334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:41.819921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc461-8a31-4e6b-805d-133abe199a5d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:41.908083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:41.908667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:41.908952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:41.909257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:41.910059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:41.912018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:41.998816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 965/1948] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:41 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:42.002742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.014387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483beb2-23c5-4714-bde7-4359c805ebbd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.029442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.031799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:42.038652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.039346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.040652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.041030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.041667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.044365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc4712-c36c-4951-892f-0a39a81f2b06 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.049902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.052241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.053063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.053349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.053617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] PATH_INFO: `/v3/projects/685378a711eb476bbf77a99253dafb01` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.055860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685378a711eb476bbf77a99253dafb01)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.058882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.077298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 984/1949] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:42.078385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.088537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:42.099708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c704-1c71-41cb-94aa-28068a8f02f1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.117580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.118222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.118506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.118783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.119560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.121203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.143165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 985/1950] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:42.149122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.172373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9197dc4-1438-4188-ad58-e8b1e775327a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.180844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.182743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.183497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.183788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] PATH_INFO: `/v3/projects/d8781891677c492dab7ce13eebb29961/users/b419b60a2cfd4bec9073fbd0fa7acfc2/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.213403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685378a711eb476bbf77a99253dafb01', 'request_id': 'req-ad18ac32-c1e5-4255-b98d-d698cfe68e20'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:42.220516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8781891677c492dab7ce13eebb29961, user_id=b419b60a2cfd4bec9073fbd0fa7acfc2, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.223720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685378a711eb476bbf77a99253dafb01', 'request_id': 'req-ad18ac32-c1e5-4255-b98d-d698cfe68e20'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:42.225152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308f835-bb8d-4430-a49a-cea52d4c9d7e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.231077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685378a711eb476bbf77a99253dafb01', 'request_id': 'req-ad18ac32-c1e5-4255-b98d-d698cfe68e20'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:42.272362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:42.308472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 986/1951] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:42 2022] PUT /identity/v3/projects/d8781891677c492dab7ce13eebb29961/users/b419b60a2cfd4bec9073fbd0fa7acfc2/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:42.314422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.314747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.315619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.329867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] The token cache is being invalidate because project 685378a711eb476bbf77a99253dafb01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:42.329867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:42.331379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ad18ac32-c1e5-4255-b98d-d698cfe68e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:42.333500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 966/1952] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:42 2022] DELETE /identity/v3/projects/685378a711eb476bbf77a99253dafb01 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:42.338364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e76dc-1c75-4138-a6fe-d8659275190a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.339289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e76dc-1c75-4138-a6fe-d8659275190a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.339991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e76dc-1c75-4138-a6fe-d8659275190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.363009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-139e76dc-1c75-4138-a6fe-d8659275190a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:42.364292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-139e76dc-1c75-4138-a6fe-d8659275190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:42.368655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] MFA Rules not processed for user `b419b60a2cfd4bec9073fbd0fa7acfc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:42.373356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:42.429918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 967/1953] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:42.437175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.437528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.437746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.461343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 987/1954] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:42.461541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:42.462460 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:42.462991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.465245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:42.478494 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 968/1955] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:42.482663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.494481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a8ec5-1176-40d4-837d-94b4cd295678 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.496334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4320eab-2c22-41a1-9aca-acb5e9f44046 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.501045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.501172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.501672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.501793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.501976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.502169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.502266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.502455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.502923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:42.503232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.503337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.506287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.511754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.541704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 969/1956] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:33:42.546919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.561673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a513bf1-30f9-4744-8075-0f7d1156d1f5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.587973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.588841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.592587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.592587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.592587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.595035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.711512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 988/1957] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/projects => generated 380 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:42.725562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.764435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 970/1958] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/projects => generated 384 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:42.770414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:42.819840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e2534-744c-4c07-bc58-cd80d6ea2be9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.874490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7de30-16b3-42f4-a161-386d84f920d6 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:42.882155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.882887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.883264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.883611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.884353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.886418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:42.907678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:42.908418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:42.908727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:42.909007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:42.909705 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:42.912535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.072132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 971/1959] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:43.076495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.087260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8ca55-14be-4b7f-a880-e358446d82ee admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.092052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.092786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.093668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.101180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.101180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.101180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.102658 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 989/1960] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:42 2022] POST /identity/v3/users => generated 297 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:43.107254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.117161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 972/1961] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:43.118056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.118381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1d486-1777-448c-989d-b421f5203a22 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.126412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.127045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.127389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.127864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.127977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22e413-cfb4-4bcd-a49e-1d5ced619795 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.128671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.130362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.133053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.133667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.138794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.138794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.138794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.138794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.141468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 990/1962] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:43.146147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.149612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 973/1963] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:43.153401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.157544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f3dad-e15d-48a3-a4df-cc567b134a5e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.163263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.163942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.164224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.164531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] PATH_INFO: `/v3/projects/3202080a0a7d455f854d5d9d8011a4a9/users/49e315c1189e436498d2fae5b24acb28/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.166150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e482-0751-4bef-97cb-247bd5394380 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.173364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.178927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.178927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.178927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] PATH_INFO: `/v3/projects/63559f266f5e43f78b08a02d18a7ff26/users/ccf000a71f2a444e9e3a612a1c9135e2/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.181536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3202080a0a7d455f854d5d9d8011a4a9, user_id=49e315c1189e436498d2fae5b24acb28, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.184230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.190945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63559f266f5e43f78b08a02d18a7ff26, user_id=ccf000a71f2a444e9e3a612a1c9135e2, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.193686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c41e1-c52f-4358-817c-105cc35a0100 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.298561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 991/1964] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:43 2022] PUT /identity/v3/projects/3202080a0a7d455f854d5d9d8011a4a9/users/49e315c1189e436498d2fae5b24acb28/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:43.302290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.335309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 974/1965] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:43 2022] PUT /identity/v3/projects/63559f266f5e43f78b08a02d18a7ff26/users/ccf000a71f2a444e9e3a612a1c9135e2/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:43.355135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.355442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.355699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.364923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a89218-ca37-4657-8e20-db52d5ed0cf4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.383175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] MFA Rules not processed for user `ccf000a71f2a444e9e3a612a1c9135e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:43.384622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:43.421071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.421716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.421995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.422268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.422944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.424585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.434434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 992/1966] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:43.438509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.449178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d526-029a-493d-b4b5-e2ff61814b32 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.450961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 975/1967] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:43.453806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.454446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.454728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.455001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] PATH_INFO: `/v3/projects/3202080a0a7d455f854d5d9d8011a4a9/users/49e315c1189e436498d2fae5b24acb28/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.455401 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.456953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3202080a0a7d455f854d5d9d8011a4a9, user_id=49e315c1189e436498d2fae5b24acb28, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.460252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d166c-7759-45d4-a9a0-9aec3f212ca1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.466442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9af645-efcf-45d9-ab92-dab7fd444b3e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.474308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 993/1968] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:43 2022] PUT /identity/v3/projects/3202080a0a7d455f854d5d9d8011a4a9/users/49e315c1189e436498d2fae5b24acb28/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:43.484126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.484462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.484736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.518509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] MFA Rules not processed for user `49e315c1189e436498d2fae5b24acb28`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:43.520321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:43.527789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.528427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.528714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.528988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.529652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.531669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.593116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 994/1969] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:43.605288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.607380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:43.658843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 976/1970] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:43 2022] POST /identity/v3/projects => generated 384 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:43.659761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.697429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b6ab9-ca1e-43c9-92a6-d78921698740 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.744167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f46d-c755-43bf-bf6a-7d99470de998 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.749424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.750343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.750700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.751135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.751818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.755805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.758729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.759429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.759788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.760601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.865170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.867077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.878836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 995/1971] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:43.935553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 977/1972] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:43 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:43.940294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.951822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21a1e-cd78-4f33-8c02-fe6a386311ab service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.956747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.958198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.958487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.958744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:43.959610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:43.961310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:43.971681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 996/1973] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:43.976523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:43.986492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81122d4-9f64-4de4-9f4f-9c81757b2459 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:43.991206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:43.991934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:43.992300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:43.992673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] PATH_INFO: `/v3/projects/e73045319d5444679c283e27d20f0a0c/users/a3b9f4373a5042cf806359ae2187d729/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.054287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73045319d5444679c283e27d20f0a0c, user_id=a3b9f4373a5042cf806359ae2187d729, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.057026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.222055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 978/1974] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:43 2022] PUT /identity/v3/projects/e73045319d5444679c283e27d20f0a0c/users/a3b9f4373a5042cf806359ae2187d729/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:44.226375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:44.238629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479f7cf-5189-41f8-ba85-b410b2d2ab54 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:44.326257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:44.326910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.327250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.327534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.328238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.330356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022e834-d98d-446e-b53a-26382af5fd27 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.343710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 997/1975] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:44 2022] GET /identity/v3/roles => generated 1427 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:44.347751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:44.360649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5d33-4641-4e3c-9d5b-fed19b369171 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:44.365734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:44.366655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.367014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.367393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] PATH_INFO: `/v3/projects/e73045319d5444679c283e27d20f0a0c/users/a3b9f4373a5042cf806359ae2187d729/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.369628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73045319d5444679c283e27d20f0a0c, user_id=a3b9f4373a5042cf806359ae2187d729, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.381059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.398623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 979/1976] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:44 2022] PUT /identity/v3/projects/e73045319d5444679c283e27d20f0a0c/users/a3b9f4373a5042cf806359ae2187d729/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:44.412877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.412877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.412877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.465117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] MFA Rules not processed for user `a3b9f4373a5042cf806359ae2187d729`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:44.466910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:44.554082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:44.556205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:44.581564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 998/1977] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:44.599664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:44.630478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4800f-752a-411d-bd7e-28186f86f790 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:44.634730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:44.635354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.635635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.640552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.640552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:44.660753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c5323-2891-4ba0-8790-b4bfb86bd876 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:44.664768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:44.665510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.665865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.666218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.677056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.678483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.697215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 999/1978] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:44.710083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.711853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.724336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:44.726603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 980/1979] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:44.855787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:44.864371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d19b-dba1-433e-a323-ff75c9c29376 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:44.868038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:44.868674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:44.868953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:44.869223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:44.928755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:44.930416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:44.940639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1000/1980] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:44.989070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:48.337767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:48.419473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a29f01-afd4-4910-8350-483bb5c0369e service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:48.424845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:48.425608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:48.426513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:48.426854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] PATH_INFO: `/v3/users/b419b60a2cfd4bec9073fbd0fa7acfc2` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:48.428521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b419b60a2cfd4bec9073fbd0fa7acfc2)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:48.430657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:48.686972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:48.695506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ec344-b64e-4c75-a05b-36945473050a service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:48.699210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:48.699786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:48.700066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:48.700333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:48.702150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:48.735694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b419b60a2cfd4bec9073fbd0fa7acfc2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:48.736421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b419b60a2cfd4bec9073fbd0fa7acfc2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:48.786464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:48.793154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e193c8-97ea-4587-9152-d46751bc4ec3 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:48.797864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1001/1981] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:48.866073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b419b60a2cfd4bec9073fbd0fa7acfc2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:48.866702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b419b60a2cfd4bec9073fbd0fa7acfc2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:48.874234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b419b60a2cfd4bec9073fbd0fa7acfc2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:48.892164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 981/1982] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:48 2022] DELETE /identity/v3/users/b419b60a2cfd4bec9073fbd0fa7acfc2 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:48.896756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:48.897048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:48.897325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:48.917595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:48.929047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:48.984710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1002/1983] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:48.994832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:48.996376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:49.056295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac197acb-13e3-42c6-9de5-3c0aaef83fcb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:49.059936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:49.060590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:49.060871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:49.061138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:49.096196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:49.096196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:49.096196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 982/1984] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:49.646999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:49.648529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:49.660513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818337b-1dea-40e0-8bf0-bd4238712bc0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:49.665305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:49.666070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:49.666446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:49.666774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] PATH_INFO: `/v3/users/86ee697e09de47b3b9c9045f1805e7d2` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:49.667995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ee697e09de47b3b9c9045f1805e7d2)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:49.670142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:49.753660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:49.757469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:49.769300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d3e9-d1b6-48d0-84ae-f674ca6b951b service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:49.777095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:49.777756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:49.778043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:49.778324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] PATH_INFO: `/v3/projects/d8781891677c492dab7ce13eebb29961` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:49.779569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8781891677c492dab7ce13eebb29961)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:49.781595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:49.924022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:49.985527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8781891677c492dab7ce13eebb29961', 'request_id': 'req-a45f2dff-7d88-4aeb-a866-361c8eb770fb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:49.989032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ee697e09de47b3b9c9045f1805e7d2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:49.989727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ee697e09de47b3b9c9045f1805e7d2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:49.994328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8781891677c492dab7ce13eebb29961', 'request_id': 'req-a45f2dff-7d88-4aeb-a866-361c8eb770fb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:49.998168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ee697e09de47b3b9c9045f1805e7d2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:49.998816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ee697e09de47b3b9c9045f1805e7d2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.001304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8781891677c492dab7ce13eebb29961', 'request_id': 'req-a45f2dff-7d88-4aeb-a866-361c8eb770fb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.014406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-79f8fe40-11bc-43cf-b41c-85529efc5d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ee697e09de47b3b9c9045f1805e7d2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.138513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] The token cache is being invalidate because project d8781891677c492dab7ce13eebb29961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:50.138955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.139699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.144559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 983/1985] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:49 2022] DELETE /identity/v3/projects/d8781891677c492dab7ce13eebb29961 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:50.152481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:50.154122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:50.244724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:50.246344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f2dff-7d88-4aeb-a866-361c8eb770fb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:50.251779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:50.252536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:50.252883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:50.253565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] PATH_INFO: `/v3/users/49e315c1189e436498d2fae5b24acb28` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:50.255349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e315c1189e436498d2fae5b24acb28)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:50.257932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:50.291426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1003/1986] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:49 2022] DELETE /identity/v3/users/86ee697e09de47b3b9c9045f1805e7d2 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:50.296350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:50.296829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:50.297101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:50.325420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:50.329319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:50.406321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1004/1987] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:50.416253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:50.417732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:50.474479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f64101-f158-4fbb-aef2-f1105fac0153 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:50.479263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:50.479979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:50.480266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:50.480578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:50.491255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:50.492727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:50.505323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1005/1988] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:50.594503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e315c1189e436498d2fae5b24acb28'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.595313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e315c1189e436498d2fae5b24acb28'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.706456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e315c1189e436498d2fae5b24acb28'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.707634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e315c1189e436498d2fae5b24acb28'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.725758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e315c1189e436498d2fae5b24acb28'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:50.735430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 984/1989] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:50 2022] DELETE /identity/v3/users/49e315c1189e436498d2fae5b24acb28 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:51.253496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:51.256270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:51.276026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6fa6a-7d3d-436a-909b-b8895c11bc63 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:51.312039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:51.314427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:51.377306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:51.377594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:51.377874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:51.378348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] PATH_INFO: `/v3/projects/1c721eb37361447cb1f210f83fb35ea7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:51.381631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c721eb37361447cb1f210f83fb35ea7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:51.384198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:51.420414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e655-25bd-4711-b95e-ed9515ce46d5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:51.426237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:51.427109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:51.427478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:51.427825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] PATH_INFO: `/v3/users/ccf000a71f2a444e9e3a612a1c9135e2` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:51.441444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf000a71f2a444e9e3a612a1c9135e2)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:51.449532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:51.603287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c721eb37361447cb1f210f83fb35ea7', 'request_id': 'req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.610997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c721eb37361447cb1f210f83fb35ea7', 'request_id': 'req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.803329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c721eb37361447cb1f210f83fb35ea7', 'request_id': 'req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.816380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] The token cache is being invalidate because project 1c721eb37361447cb1f210f83fb35ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:51.816994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.817877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.820060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1006/1990] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:51 2022] DELETE /identity/v3/projects/1c721eb37361447cb1f210f83fb35ea7 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:51.820780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:51.822198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:51.853995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf000a71f2a444e9e3a612a1c9135e2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.856571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf000a71f2a444e9e3a612a1c9135e2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:51.900208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a11a01-c29b-4d28-a4d3-450fa5a0fee1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:51.908766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:51.909792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:51.911297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:51.911577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] PATH_INFO: `/v3/projects/3202080a0a7d455f854d5d9d8011a4a9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:51.914106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3202080a0a7d455f854d5d9d8011a4a9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:51.917810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:52.067453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf000a71f2a444e9e3a612a1c9135e2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.069524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf000a71f2a444e9e3a612a1c9135e2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.078658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf000a71f2a444e9e3a612a1c9135e2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.103091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 985/1991] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:51 2022] DELETE /identity/v3/users/ccf000a71f2a444e9e3a612a1c9135e2 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:52.104130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:52.105681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:52.146756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3202080a0a7d455f854d5d9d8011a4a9', 'request_id': 'req-dfd4f768-163f-4454-98c0-78f2cdff34b4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.153194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3202080a0a7d455f854d5d9d8011a4a9', 'request_id': 'req-dfd4f768-163f-4454-98c0-78f2cdff34b4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.185640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855f77c-d63d-4602-9dfa-1423f3948cd6 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:52.242543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:52.243440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.243735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.244019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] PATH_INFO: `/v3/users/1af617629ce34eedbf9ff51ab9a77174` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.245291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af617629ce34eedbf9ff51ab9a77174)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:52.247276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:52.326062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3202080a0a7d455f854d5d9d8011a4a9', 'request_id': 'req-dfd4f768-163f-4454-98c0-78f2cdff34b4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.341853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] The token cache is being invalidate because project 3202080a0a7d455f854d5d9d8011a4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:52.342243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.342975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dfd4f768-163f-4454-98c0-78f2cdff34b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.344657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1007/1992] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:51 2022] DELETE /identity/v3/projects/3202080a0a7d455f854d5d9d8011a4a9 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:52.345937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313631f4-137b-4b45-8bc8-de92175003dd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.346233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313631f4-137b-4b45-8bc8-de92175003dd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.346514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313631f4-137b-4b45-8bc8-de92175003dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.365562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-313631f4-137b-4b45-8bc8-de92175003dd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:52.367006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-313631f4-137b-4b45-8bc8-de92175003dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:52.375389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1008/1993] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:52.376774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.377139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.377492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.397686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:52.398958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:52.406938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1009/1994] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:52.407642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:52.408934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:52.466748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832a727-0f42-411e-a02f-156bc07d7807 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:52.512189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:52.512189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.512189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.512189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d637c-ac0c-4035-ab03-7e19f803c1a4 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1010/1995] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.513197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.513593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:52.513593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:52.522115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1011/1996] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:52.526248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:52.535235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21646-f847-4262-b722-b5d885127fe1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:52.539920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:52.540565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.540848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.541121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.541810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:52.542115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:52.543706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:52.555776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1012/1997] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:33:52.560046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:52.570665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9ee2-1d56-4c39-a407-e91133e787c3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:52.575659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:52.576284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.576618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.577408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.578133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:52.580078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:52.659771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af617629ce34eedbf9ff51ab9a77174'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.659771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af617629ce34eedbf9ff51ab9a77174'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.737070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1013/1998] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:52 2022] POST /identity/v3/projects => generated 376 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:52.744958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:52.769016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af617629ce34eedbf9ff51ab9a77174'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.770510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af617629ce34eedbf9ff51ab9a77174'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.778799 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af617629ce34eedbf9ff51ab9a77174'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:52.800748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 986/1999] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:52 2022] DELETE /identity/v3/users/1af617629ce34eedbf9ff51ab9a77174 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:52.833051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596b9c-165e-49af-bf6c-e70467f93115 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:52.896394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:52.897921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:52.897921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:52.897921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:52.898346 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:52.900446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.147140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1014/2000] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:52 2022] POST /identity/v3/users => generated 289 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:53.151353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.152817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:53.165559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268d41-0e29-49f1-bddf-7352153b5241 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.170063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.170674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.170961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.171240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.171946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.173564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.183348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 987/2001] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:53 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:53.187600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.198063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651515f-864e-4df1-9a3b-2af735c7f412 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.203340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.203340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.203531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.203737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.204448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.206034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.216943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1015/2002] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:53.221174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.231371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad50a9-2462-4817-9dcf-990ec8838865 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.235981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.236677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.236962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.237236 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] PATH_INFO: `/v3/projects/c0de47f14baa469097cacedce5badd10/users/904a6b62111842a1b81ae2cbf99a3a7f/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.251941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0de47f14baa469097cacedce5badd10, user_id=904a6b62111842a1b81ae2cbf99a3a7f, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.260909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588afdef-50f0-44d8-b019-d2dc639688b1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.263558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.265357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:53.279359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ea932-a143-4623-b9a0-1248132e9755 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.284342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.285005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.285345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.285623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] PATH_INFO: `/v3/projects/63559f266f5e43f78b08a02d18a7ff26` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.286772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63559f266f5e43f78b08a02d18a7ff26)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.288845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.404171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 988/2003] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:53 2022] PUT /identity/v3/projects/c0de47f14baa469097cacedce5badd10/users/904a6b62111842a1b81ae2cbf99a3a7f/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:53.415204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.415478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.416248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.438750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63559f266f5e43f78b08a02d18a7ff26', 'request_id': 'req-c5615fa0-5391-4024-88fa-36517dfd8f54'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.445145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] MFA Rules not processed for user `904a6b62111842a1b81ae2cbf99a3a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:53.445374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63559f266f5e43f78b08a02d18a7ff26', 'request_id': 'req-c5615fa0-5391-4024-88fa-36517dfd8f54'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.446370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:53.510189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 989/2004] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:53.514642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.574431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228a650-334a-4682-ae54-7335cb5da687 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.579019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.579657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.579918 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.580196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.580899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.582796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.615733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63559f266f5e43f78b08a02d18a7ff26', 'request_id': 'req-c5615fa0-5391-4024-88fa-36517dfd8f54'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.620360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] The token cache is being invalidate because project 63559f266f5e43f78b08a02d18a7ff26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:53.620782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.621408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.622952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1016/2005] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:53 2022] DELETE /identity/v3/projects/63559f266f5e43f78b08a02d18a7ff26 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:53.624080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.625537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:53.638881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615fa0-5391-4024-88fa-36517dfd8f54 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.644327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.645215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.645796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.646193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] PATH_INFO: `/v3/projects/51f963e7bae9451da70be719e34416ab` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.647460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f963e7bae9451da70be719e34416ab)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.649465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:53.798793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 990/2006] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:53 2022] POST /identity/v3/projects => generated 376 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:53.799748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:53.801374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:53.837310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f963e7bae9451da70be719e34416ab', 'request_id': 'req-b3bdc09a-20a1-4138-a104-e5f10a0bd503'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.843963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f963e7bae9451da70be719e34416ab', 'request_id': 'req-b3bdc09a-20a1-4138-a104-e5f10a0bd503'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:53.866524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34431-7153-461a-831e-5f50f45084e3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:53.923390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:53.924030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:53.924313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:53.924622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] PATH_INFO: `/v3/users/a3b9f4373a5042cf806359ae2187d729` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:53.925887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b9f4373a5042cf806359ae2187d729)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:53.927769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.078305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f963e7bae9451da70be719e34416ab', 'request_id': 'req-b3bdc09a-20a1-4138-a104-e5f10a0bd503'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.090248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] The token cache is being invalidate because project 51f963e7bae9451da70be719e34416ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:54.090392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.090959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.092493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1017/2007] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:53 2022] DELETE /identity/v3/projects/51f963e7bae9451da70be719e34416ab => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:54.093258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.094520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.107674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdc09a-20a1-4138-a104-e5f10a0bd503 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.114114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.114114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.114114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.114416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.115084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.117084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c32176-be73-4041-921d-73f075b93172 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.209805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:54.231761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1018/2008] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:54.233045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.233331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.233610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.240725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b9f4373a5042cf806359ae2187d729'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.241395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b9f4373a5042cf806359ae2187d729'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.252767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:54.253993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.262096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1019/2009] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:54.262748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.272888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602725fc-21b4-4bbc-9872-3bfd05e95174 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.277439 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.278042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.278336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.278602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.279296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.280957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8f65f-b840-4668-9227-0a3170ff3970 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.291495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1020/2010] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:54.292669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f25d-502e-4393-a719-818531de399f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.292958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f25d-502e-4393-a719-818531de399f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.293232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f25d-502e-4393-a719-818531de399f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.311400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0030f25d-502e-4393-a719-818531de399f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:54.312634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0030f25d-502e-4393-a719-818531de399f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.320497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1021/2011] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:54.321236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030f25d-502e-4393-a719-818531de399f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.330125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030f25d-502e-4393-a719-818531de399f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.335224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.335846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.336130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.336434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] PATH_INFO: `/v3/projects/77753d3971fe4f49a0af458969daacc2/users/605bc9e624994064aadbee1fc4fccc12/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.346059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b9f4373a5042cf806359ae2187d729'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.346845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b9f4373a5042cf806359ae2187d729'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.351185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77753d3971fe4f49a0af458969daacc2, user_id=605bc9e624994064aadbee1fc4fccc12, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.354010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.355413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b9f4373a5042cf806359ae2187d729'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:54.373890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 991/2012] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:53 2022] DELETE /identity/v3/users/a3b9f4373a5042cf806359ae2187d729 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:54.374770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.379478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.447234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5a679-8ae7-4011-bfc9-d59eeeb91745 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.451960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.452614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.452901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.453177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.453996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:54.454316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.455917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.467674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 992/2013] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:33:54.473346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.483609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc55e-7d95-4a5d-a0c0-f715ed076f49 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.488969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.489581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.489863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.490166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.490834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.492761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.500576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1022/2014] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:54 2022] PUT /identity/v3/projects/77753d3971fe4f49a0af458969daacc2/users/605bc9e624994064aadbee1fc4fccc12/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:54.505084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.506769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.571609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba999a-2749-40b9-bb89-bba7c539ad47 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.576342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.576984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.577263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.577533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.578642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.580233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.590948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1023/2015] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:54.596139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.608778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6353-c00c-4a10-9226-b9ddb3108b6d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.614115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.614847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.615092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.615360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] PATH_INFO: `/v3/projects/77753d3971fe4f49a0af458969daacc2/users/605bc9e624994064aadbee1fc4fccc12/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.617389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=77753d3971fe4f49a0af458969daacc2, user_id=605bc9e624994064aadbee1fc4fccc12, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.620032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7652de-9d97-478b-a417-153f7281c35d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.631150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1024/2016] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:54 2022] PUT /identity/v3/projects/77753d3971fe4f49a0af458969daacc2/users/605bc9e624994064aadbee1fc4fccc12/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:54.642578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.642874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.643158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.660926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 993/2017] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/projects => generated 370 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:54.662120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.676975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] MFA Rules not processed for user `605bc9e624994064aadbee1fc4fccc12`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:54.678297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.727360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ac2b4-a5d3-430c-8646-28176e07291b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.743550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.744187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.744501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.744777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.745490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.747416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.754337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:54.772134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1025/2018] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:54.793778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.795281 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.851563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a33c-e047-4806-a30d-f9f12af3028a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.855341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.856648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.856935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.857214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.859013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:54.926895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.929968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.939710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 994/2019] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:54 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:33:54.940934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1026/2020] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:54.947734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.957873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d668c-14a9-4306-9113-95bd58dabb5e admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.962501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.963108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.963389 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.963659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:54.964348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:54.966053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:54.977284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 995/2021] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:54.981493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:54.993273 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0cafc-0ea6-49ab-8e6b-16a48b5b3ae6 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:54.998485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:54.999201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:54.999482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:54.999759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.000515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.002155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.016759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1027/2022] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:54 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:33:55.018727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.022550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.025057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:55.044074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea30b41-d3ed-4055-be88-e9931bb85c01 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.046567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01431eb4-fecb-4fea-8df0-582f1bdde342 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.053501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.054537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.055392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.055798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] PATH_INFO: `/v3/projects/3e41fbf84b7a44a198e9363b2caf2d25/users/1786fffd47c54049b54e381a705dac4c/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.055913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.056811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.057160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.057661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] PATH_INFO: `/v3/projects/e73045319d5444679c283e27d20f0a0c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.059715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73045319d5444679c283e27d20f0a0c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.062567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.074062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e41fbf84b7a44a198e9363b2caf2d25, user_id=1786fffd47c54049b54e381a705dac4c, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.077844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55940ad-3854-46dd-9672-f6bf16f257f3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.340676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 996/2023] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:55 2022] PUT /identity/v3/projects/3e41fbf84b7a44a198e9363b2caf2d25/users/1786fffd47c54049b54e381a705dac4c/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:55.352597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.352875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.353366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.357259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73045319d5444679c283e27d20f0a0c', 'request_id': 'req-bb65981c-e1e8-4aa9-9778-8edcc8aff497'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:55.364381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73045319d5444679c283e27d20f0a0c', 'request_id': 'req-bb65981c-e1e8-4aa9-9778-8edcc8aff497'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:55.383092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] MFA Rules not processed for user `1786fffd47c54049b54e381a705dac4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:55.384381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:55.451352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 997/2024] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:33:55.455729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.478174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73045319d5444679c283e27d20f0a0c', 'request_id': 'req-bb65981c-e1e8-4aa9-9778-8edcc8aff497'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:55.497221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] The token cache is being invalidate because project e73045319d5444679c283e27d20f0a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:55.497221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:55.497807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb65981c-e1e8-4aa9-9778-8edcc8aff497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:55.499608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1028/2025] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:55 2022] DELETE /identity/v3/projects/e73045319d5444679c283e27d20f0a0c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:55.505458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8afcdb-b5dd-4626-834e-f9e9b17acb6b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.506005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8afcdb-b5dd-4626-834e-f9e9b17acb6b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.506127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8afcdb-b5dd-4626-834e-f9e9b17acb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.529926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1e8afcdb-b5dd-4626-834e-f9e9b17acb6b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:55.531101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1e8afcdb-b5dd-4626-834e-f9e9b17acb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:55.546919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7484a00-638d-41a3-bff2-0f2ad0909307 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.569798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.569798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.569798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.569798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.570093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.570093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.594822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1029/2026] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:55.602788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617876f-5a76-424e-8343-e62a46905371 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.603081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617876f-5a76-424e-8343-e62a46905371 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.603391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617876f-5a76-424e-8343-e62a46905371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.622218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9617876f-5a76-424e-8343-e62a46905371 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:55.623438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9617876f-5a76-424e-8343-e62a46905371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:55.632307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1030/2027] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:55.636348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617876f-5a76-424e-8343-e62a46905371 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.646133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617876f-5a76-424e-8343-e62a46905371 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.651028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.651654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.651935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.652205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.652931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:33:55.653237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.654858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.667327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1031/2028] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:33:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:33:55.671550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.681789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888d353-da42-479f-a35e-f572d9289fe8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.686456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.687055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.687326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.693810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.693810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.693810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.787097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 998/2029] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/projects => generated 368 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:55.787097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.789015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:55.852297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200ea7-2e31-4bf0-8697-3affe95f230d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.857281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.857904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.858188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.858464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.859133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.861089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:55.897251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1032/2030] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/projects => generated 372 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:55.901274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:55.959267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df5dfe-c063-4f96-9ef6-56c0fdbe13a6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:55.963967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:55.973719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:55.973719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:55.973719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:55.973719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:55.973719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.025922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 999/2031] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:56.025922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.030495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24fc77-e3f6-4c83-a0b5-e65ca63df987 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.035206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.036365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.036704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.036982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.037691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.039279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.050077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1000/2032] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:56.054721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.064620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038caf7-7c76-4800-95ef-2f9f07b8990c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.077156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.078270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.078827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.079298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] PATH_INFO: `/v3/projects/57faf360e635488c8744b7399c53802e/users/831a298ff43d4d9084562dcd29ac23b6/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.099162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57faf360e635488c8744b7399c53802e, user_id=831a298ff43d4d9084562dcd29ac23b6, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.102455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.146931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1033/2033] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:55 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:56.150863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.161049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391e803-e8db-4086-a71a-f1e6131a481d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.165838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.166429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.166710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.166984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.167731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.169387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.180463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1034/2034] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:56.184316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.194284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728842d-7668-4f3a-b85c-7b452afa97e9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.199405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.199995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.200274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.200577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.201249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.202804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.217173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1035/2035] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:56.218263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.228776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dcea5-225d-4153-9cbc-58bc1e55b0e2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.233803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.234460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.234744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.235018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] PATH_INFO: `/v3/projects/686d573bd8724d0a92250816f3432d2e/users/15d0b7566d684afa871672aabb17c2cc/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.250381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=686d573bd8724d0a92250816f3432d2e, user_id=15d0b7566d684afa871672aabb17c2cc, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.253064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24d940-cd51-41b8-9a0b-b83cb6724fc3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.290017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1001/2036] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:56 2022] PUT /identity/v3/projects/57faf360e635488c8744b7399c53802e/users/831a298ff43d4d9084562dcd29ac23b6/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:56.293778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.354629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0961ec-fe16-4898-a241-f8627c16f2d2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.359438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.360061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.360347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.360725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.361439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.363395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.374107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1002/2037] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:56.378151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.389722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bb356-b7eb-4b32-b518-ec178c2cdb89 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.396731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.397423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.397706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.398157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] PATH_INFO: `/v3/projects/57faf360e635488c8744b7399c53802e/users/831a298ff43d4d9084562dcd29ac23b6/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.400379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57faf360e635488c8744b7399c53802e, user_id=831a298ff43d4d9084562dcd29ac23b6, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.404562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11940dd0-6c46-49e2-8cb5-4ed4560ac7b7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.410934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1036/2038] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:56 2022] PUT /identity/v3/projects/686d573bd8724d0a92250816f3432d2e/users/15d0b7566d684afa871672aabb17c2cc/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:56.414783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1003/2039] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:56 2022] PUT /identity/v3/projects/57faf360e635488c8744b7399c53802e/users/831a298ff43d4d9084562dcd29ac23b6/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:56.416063 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.416412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.417321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.426666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.426971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.427257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.444855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:56.444855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:56.465444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] MFA Rules not processed for user `831a298ff43d4d9084562dcd29ac23b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:56.466766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:56.506985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1037/2040] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:56.521118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.522501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:56.547456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1004/2041] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:56.558768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:56.565283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:56.579496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21598c04-2cee-479e-bccf-bc9a59eddc61 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.583327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.583971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.584256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.584631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.593908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.595335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.604926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1038/2042] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:56.617028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48812328-007e-4fb7-af4c-164b3e1ae66c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:56.620874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:56.621505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:56.621787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:56.622060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:56.632308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:56.633768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e634b5-17d0-43c7-a6fe-31524b5ba567 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:56.643537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1005/2043] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:33:57.197939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:57.221320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:57.222774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:57.235674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe24b7-830e-4c96-b920-15401680ab00 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:57.241167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:57.241797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:57.242076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:57.242400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] PATH_INFO: `/v3/users/a92fb0ef6047443da1de4de87f092ea9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:57.243581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92fb0ef6047443da1de4de87f092ea9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:57.245500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:57.560796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92fb0ef6047443da1de4de87f092ea9'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:57.561520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92fb0ef6047443da1de4de87f092ea9'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:57.569385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92fb0ef6047443da1de4de87f092ea9'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:57.570104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92fb0ef6047443da1de4de87f092ea9'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:57.583075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92fb0ef6047443da1de4de87f092ea9'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:57.706452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:33:57.712697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1039/2044] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:57 2022] DELETE /identity/v3/users/a92fb0ef6047443da1de4de87f092ea9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:57.723397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b420e-bd57-42fa-8458-4b99166048ce None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:57.723689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b420e-bd57-42fa-8458-4b99166048ce None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:57.723969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b420e-bd57-42fa-8458-4b99166048ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:57.743466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3e5b420e-bd57-42fa-8458-4b99166048ce None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:57.744869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3e5b420e-bd57-42fa-8458-4b99166048ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:57.808209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1006/2045] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:57.817842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:57.819456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:57.876843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896961c-cc8b-4422-8ff3-8f638d712112 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:57.880526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:57.881206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:57.881423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:57.881699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:57.891191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:57.892649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:57.902460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1040/2046] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:33:58.952270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:58.952270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:58.952270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:58.953971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:58.960495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:58.975195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a906c-9285-46b9-a6c1-fee2566cb3ff service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:58.984368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] MFA Rules not processed for user `15d0b7566d684afa871672aabb17c2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:33:58.985731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:59.000817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.001682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.001968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.002247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] PATH_INFO: `/v3/projects/5d2cb91725814907950373291fdfa528` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.003482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2cb91725814907950373291fdfa528)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.006380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.053452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1007/2047] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:33:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:33:59.057887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.063556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:59.072638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2edfe1-ee5d-426b-a8e9-a381918ab077 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.077198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.077794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.078080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.078359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.079021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.081016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.141677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2cb91725814907950373291fdfa528', 'request_id': 'req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.148571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2cb91725814907950373291fdfa528', 'request_id': 'req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.296699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1008/2048] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:33:59 2022] POST /identity/v3/projects => generated 372 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:59.301063 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.341550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2cb91725814907950373291fdfa528', 'request_id': 'req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.355365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] The token cache is being invalidate because project 5d2cb91725814907950373291fdfa528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:33:59.355664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.356358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.358360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1041/2049] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:33:58 2022] DELETE /identity/v3/projects/5d2cb91725814907950373291fdfa528 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:33:59.362482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.362841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87d886-bd5c-4163-9bec-4c9ee241bbe2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.365130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:59.369628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.370566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.371009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.371605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.372516 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.374949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.432848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1702d2-e2c5-4a5a-9cb0-a6e8f1faa3de admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.437507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.438143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.438431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.438706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] PATH_INFO: `/v3/users/52654794fb45493b841676476bc4f494` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.439863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=52654794fb45493b841676476bc4f494)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.441841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.530440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1009/2050] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:33:59 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:33:59.534231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.536812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:59.551399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bf1ea-805c-4377-8164-f7ef570de156 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.556619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.557429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.557795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.558076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.558975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.560729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.573527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1010/2051] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:59 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:59.577829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.587681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339edb47-473c-49ac-a02f-c62782d33c7a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.592316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.592994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.593276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.593550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] PATH_INFO: `/v3/projects/a0dd5a7a647a465b852cad5b51cbde56/users/c124f87901294a4b992adf8a2691a423/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.607767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dd5a7a647a465b852cad5b51cbde56, user_id=c124f87901294a4b992adf8a2691a423, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.610439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.730666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1011/2052] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:59 2022] PUT /identity/v3/projects/a0dd5a7a647a465b852cad5b51cbde56/users/c124f87901294a4b992adf8a2691a423/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:59.734326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.746100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52654794fb45493b841676476bc4f494'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.746998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52654794fb45493b841676476bc4f494'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.755477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52654794fb45493b841676476bc4f494'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.756373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52654794fb45493b841676476bc4f494'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.776347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52654794fb45493b841676476bc4f494'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:33:59.810531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b872db-7200-4959-a444-2862af5acb24 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.815455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31929189-a889-4010-b649-0183e888b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.816070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31929189-a889-4010-b649-0183e888b78d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.816400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31929189-a889-4010-b649-0183e888b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.816684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31929189-a889-4010-b649-0183e888b78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.817387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31929189-a889-4010-b649-0183e888b78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.819004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31929189-a889-4010-b649-0183e888b78d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.828928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1012/2053] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:33:59 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:33:59.829732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31929189-a889-4010-b649-0183e888b78d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.831636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-31929189-a889-4010-b649-0183e888b78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:33:59.845309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31929189-a889-4010-b649-0183e888b78d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:33:59.851423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:33:59.852076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:33:59.852360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:33:59.852669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] PATH_INFO: `/v3/users/904a6b62111842a1b81ae2cbf99a3a7f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:33:59.853847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=904a6b62111842a1b81ae2cbf99a3a7f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:33:59.855763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:33:59.936416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1042/2054] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:59 2022] DELETE /identity/v3/users/52654794fb45493b841676476bc4f494 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:33:59.938124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:33:59.945769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:00.028880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c2f9-c640-4b17-9753-05f49a60b4ff admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:00.033862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:00.034527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:00.034826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:00.035126 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] PATH_INFO: `/v3/projects/a0dd5a7a647a465b852cad5b51cbde56/users/c124f87901294a4b992adf8a2691a423/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:00.037726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dd5a7a647a465b852cad5b51cbde56, user_id=c124f87901294a4b992adf8a2691a423, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:00.040839 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03436392-28b9-495a-84b8-a8c13f9c6d6c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:00.052044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1043/2055] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:33:59 2022] PUT /identity/v3/projects/a0dd5a7a647a465b852cad5b51cbde56/users/c124f87901294a4b992adf8a2691a423/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:00.200013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904a6b62111842a1b81ae2cbf99a3a7f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.200874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904a6b62111842a1b81ae2cbf99a3a7f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.334735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904a6b62111842a1b81ae2cbf99a3a7f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.335366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904a6b62111842a1b81ae2cbf99a3a7f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.343097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904a6b62111842a1b81ae2cbf99a3a7f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.363940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1013/2056] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:33:59 2022] DELETE /identity/v3/users/904a6b62111842a1b81ae2cbf99a3a7f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:00.368416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:00.368785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:00.369133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:00.388490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:00.389717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:00.457763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1044/2057] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:00.467402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:00.472331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:00.531982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed91a-333f-491b-b62a-f335350a6a7d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:00.536062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:00.536896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:00.537296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:00.537638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:00.547214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:00.548667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:00.559996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:00.561507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:00.564729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1014/2058] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:00.575137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b86de-d609-45b3-882e-1c5dfdc1d5ba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:00.580037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:00.580949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:00.581072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:00.581266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] PATH_INFO: `/v3/projects/28457df82331453dbf9491cc3c079351` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:00.582456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28457df82331453dbf9491cc3c079351)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:00.584697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:00.744536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28457df82331453dbf9491cc3c079351', 'request_id': 'req-0adb0d10-7197-42a0-8f74-0bca0ef82107'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.750956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28457df82331453dbf9491cc3c079351', 'request_id': 'req-0adb0d10-7197-42a0-8f74-0bca0ef82107'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.906844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28457df82331453dbf9491cc3c079351', 'request_id': 'req-0adb0d10-7197-42a0-8f74-0bca0ef82107'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.921097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] The token cache is being invalidate because project 28457df82331453dbf9491cc3c079351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:00.921367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.922136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:00.924153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1045/2059] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:00 2022] DELETE /identity/v3/projects/28457df82331453dbf9491cc3c079351 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:00.928478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:00.939262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33faa4b1-a479-4439-802e-3cd06b032923 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:01.001162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:01.001725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:01.002002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:01.002274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] PATH_INFO: `/v3/users/1c3e248a4e2d4a5bbd3965fab30208f0` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:01.003433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3e248a4e2d4a5bbd3965fab30208f0)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:01.006021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:01.117871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:01.288118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:01.289593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:01.305225 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb0d10-7197-42a0-8f74-0bca0ef82107 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:01.309859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:01.310494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:01.310792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:01.311064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] PATH_INFO: `/v3/projects/c0de47f14baa469097cacedce5badd10` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:01.312211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0de47f14baa469097cacedce5badd10)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:01.314176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:01.420228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3e248a4e2d4a5bbd3965fab30208f0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.421442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3e248a4e2d4a5bbd3965fab30208f0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.462984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0de47f14baa469097cacedce5badd10', 'request_id': 'req-1b738c14-6988-4b7a-b09e-fc6b6a686dca'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.469650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0de47f14baa469097cacedce5badd10', 'request_id': 'req-1b738c14-6988-4b7a-b09e-fc6b6a686dca'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.632304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3e248a4e2d4a5bbd3965fab30208f0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.632963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3e248a4e2d4a5bbd3965fab30208f0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.642489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3e248a4e2d4a5bbd3965fab30208f0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.663404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1015/2060] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:00 2022] DELETE /identity/v3/users/1c3e248a4e2d4a5bbd3965fab30208f0 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:01.664559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:01.670368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:01.743816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69354c33-2ae6-4d81-8580-60195efe8aa8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:01.748501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:01.749231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:01.749578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:01.749920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] PATH_INFO: `/v3/users/1786fffd47c54049b54e381a705dac4c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:01.751152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1786fffd47c54049b54e381a705dac4c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:01.753161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:01.769060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0de47f14baa469097cacedce5badd10', 'request_id': 'req-1b738c14-6988-4b7a-b09e-fc6b6a686dca'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.779788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] The token cache is being invalidate because project c0de47f14baa469097cacedce5badd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:01.780101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.780776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:01.782291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1046/2061] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:01 2022] DELETE /identity/v3/projects/c0de47f14baa469097cacedce5badd10 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:01.786253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:01.787605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:01.849708 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b738c14-6988-4b7a-b09e-fc6b6a686dca admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:01.855103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:01.855738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:01.856021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:01.856297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] PATH_INFO: `/v3/users/605bc9e624994064aadbee1fc4fccc12` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:01.857498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] RBAC: Authorizing `identity:delete_user(user_id=605bc9e624994064aadbee1fc4fccc12)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:01.859405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:02.180515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:02.206802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1786fffd47c54049b54e381a705dac4c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.208034 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1786fffd47c54049b54e381a705dac4c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.395269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1786fffd47c54049b54e381a705dac4c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.407141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1786fffd47c54049b54e381a705dac4c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.416938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1786fffd47c54049b54e381a705dac4c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.420501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605bc9e624994064aadbee1fc4fccc12'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.421348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605bc9e624994064aadbee1fc4fccc12'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.431191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605bc9e624994064aadbee1fc4fccc12'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.432155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605bc9e624994064aadbee1fc4fccc12'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.436491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1016/2062] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:01 2022] DELETE /identity/v3/users/1786fffd47c54049b54e381a705dac4c => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:02.437476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:02.442267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.447710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7edc16b5-2a1c-488e-a1d1-b61252d8d59f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605bc9e624994064aadbee1fc4fccc12'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.514161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd37ef-376a-436c-9b36-e2f94435fb11 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:02.519025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:02.519685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:02.519971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:02.520238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] PATH_INFO: `/v3/projects/b09ad34296d0497ab7d3a4842e174fd0` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:02.521417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09ad34296d0497ab7d3a4842e174fd0)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:02.523373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:02.603264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1047/2063] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:01 2022] DELETE /identity/v3/users/605bc9e624994064aadbee1fc4fccc12 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:02.604575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b3000-ec90-407b-853c-455683dbcae4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:02.604863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b3000-ec90-407b-853c-455683dbcae4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:02.605141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b3000-ec90-407b-853c-455683dbcae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:02.634359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-634b3000-ec90-407b-853c-455683dbcae4 None None] MFA Rules not processed for user `c124f87901294a4b992adf8a2691a423`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:02.635578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-634b3000-ec90-407b-853c-455683dbcae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.713485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1048/2064] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:02.715907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:02.716199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:02.716505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:02.734799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:02.736003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.780597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09ad34296d0497ab7d3a4842e174fd0', 'request_id': 'req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.787174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09ad34296d0497ab7d3a4842e174fd0', 'request_id': 'req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.792701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09ad34296d0497ab7d3a4842e174fd0', 'request_id': 'req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.793432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1049/2065] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:02.794270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:02.795534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.851345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc58e-c299-4ce8-b5d4-23b9a92e2eb9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:02.898013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:02.898625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:02.898913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:02.899189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:02.900976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.960202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:02.961692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:02.970488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] The token cache is being invalidate because project b09ad34296d0497ab7d3a4842e174fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:02.970852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.971612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-675ffe77-8ebd-4f7d-b9fb-98dc9458466e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:02.972146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1050/2066] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:02.973885 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:02.975331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:02.983252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1017/2067] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:02 2022] DELETE /identity/v3/projects/b09ad34296d0497ab7d3a4842e174fd0 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:02.987971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff46b0-cf37-4de1-8884-0ddb88fe91a8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:02.988624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff46b0-cf37-4de1-8884-0ddb88fe91a8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:02.988624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff46b0-cf37-4de1-8884-0ddb88fe91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.011167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6bff46b0-cf37-4de1-8884-0ddb88fe91a8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:03.012403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6bff46b0-cf37-4de1-8884-0ddb88fe91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.039421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe3a6e-a644-485b-99d4-568f4b02baef service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.044852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.045484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.045771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.046143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.048029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.088597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1018/2068] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:03.094069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.094607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.095062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.118854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:03.120311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.125434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.126878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.132548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1019/2069] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:03.133728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.136920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1051/2070] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:03.146687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcf21a-cdda-4ede-84a2-f551be622c9d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.155463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.155463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.155463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.155463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.155825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:03.155825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.156622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.168979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1020/2071] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:03.173263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.183559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e066f-8ea0-4c41-8b47-d5947ac02fc9 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.188176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.188823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.189108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.189386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.207487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.207487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.247398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.247398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.274645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc72c2-e661-44a2-84e4-928f0cf514d1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.285948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.286801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.287090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.288073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] PATH_INFO: `/v3/projects/77753d3971fe4f49a0af458969daacc2` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.289557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77753d3971fe4f49a0af458969daacc2)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.291980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.372937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1052/2072] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/projects => generated 376 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:34:03.374487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.431202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77753d3971fe4f49a0af458969daacc2', 'request_id': 'req-74f26db1-d113-4ef7-a627-2966bae93b98'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:03.438875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77753d3971fe4f49a0af458969daacc2', 'request_id': 'req-74f26db1-d113-4ef7-a627-2966bae93b98'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:03.445840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77753d3971fe4f49a0af458969daacc2', 'request_id': 'req-74f26db1-d113-4ef7-a627-2966bae93b98'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:03.446394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574d9a7-40b2-4f54-9b7c-1787238caa1e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.522553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.523820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.524111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.524401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.525075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.527168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.541612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] The token cache is being invalidate because project 77753d3971fe4f49a0af458969daacc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:03.546126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:03.546126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-74f26db1-d113-4ef7-a627-2966bae93b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:03.552506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1021/2073] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:03 2022] DELETE /identity/v3/projects/77753d3971fe4f49a0af458969daacc2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:03.554500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77688a-cb75-4195-9878-c9a767999c6f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.555061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77688a-cb75-4195-9878-c9a767999c6f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.555448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77688a-cb75-4195-9878-c9a767999c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.588661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5d77688a-cb75-4195-9878-c9a767999c6f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:03.589901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5d77688a-cb75-4195-9878-c9a767999c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.616459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1022/2074] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:03.621132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.621498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.621839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.661814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:03.663547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.677930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1023/2075] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:03.678852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.686621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1053/2076] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/users => generated 289 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:34:03.687489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.689325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:03.689660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006bbf-e887-4c5c-b40e-e59150e279d5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.694511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.695136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.695422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.695843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.709885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:03.709885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.709885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.710322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0335a-ccfa-43b4-bff5-dcc7309da2f0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.710322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.710322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.710322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.710610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.710610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.711714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.712433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1024/2077] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:03.715788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.725909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1054/2078] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:03 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:03.729731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.734065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcf0ae-8a21-4a40-9122-780d5dc699cc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.740610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece600d-13ea-4e8d-810c-f3b50bb67c93 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.746907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.747455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.747593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.747805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.748061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.748170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.748358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.748636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.748789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.749322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.750655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.750942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.763026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1055/2079] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:03 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:03.769655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.780172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f928-897d-489b-801e-9b2eb77d9b5d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:03.784951 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:03.786180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:03.786464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:03.786814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] PATH_INFO: `/v3/projects/49f71964c1af45649fbe7154f30f298c/users/b25da672e758453597c14eeff3184947/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:03.802157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f71964c1af45649fbe7154f30f298c, user_id=b25da672e758453597c14eeff3184947, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:03.804840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:03.920595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1025/2080] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:03 2022] POST /identity/v3/projects => generated 398 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:03.921960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:03.925320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.001144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604e04-b71d-476e-9040-3e4b54d27e0c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.005933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.006583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.006862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.007134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] PATH_INFO: `/v3/projects/3e41fbf84b7a44a198e9363b2caf2d25` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.008268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e41fbf84b7a44a198e9363b2caf2d25)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.010203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.043344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1056/2081] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:03 2022] PUT /identity/v3/projects/49f71964c1af45649fbe7154f30f298c/users/b25da672e758453597c14eeff3184947/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:04.044153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.107947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd02ae-fef8-44f1-a038-e92d216bd8ca admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.112551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.113764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.114250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.114696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.115735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.117796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c65a2-38c2-4866-8ff8-50b571551883 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.217663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e41fbf84b7a44a198e9363b2caf2d25', 'request_id': 'req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.224102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e41fbf84b7a44a198e9363b2caf2d25', 'request_id': 'req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.229681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e41fbf84b7a44a198e9363b2caf2d25', 'request_id': 'req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.289445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1057/2082] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:04 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:04.290895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.291193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.291447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.318789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] MFA Rules not processed for user `b25da672e758453597c14eeff3184947`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:04.319987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.329536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:04.393612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1058/2083] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:04.394472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.432242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] The token cache is being invalidate because project 3e41fbf84b7a44a198e9363b2caf2d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:04.432650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.433471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.435394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1026/2084] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:03 2022] DELETE /identity/v3/projects/3e41fbf84b7a44a198e9363b2caf2d25 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:04.436518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.439129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.460208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baf855-47b3-47f1-971a-6bf997103cbe None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.466584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.467316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.468235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.468546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.469310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.471680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.483095 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1059/2085] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:04 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:04.489056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.489056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.501128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c537e-1c5d-4201-b890-82584e836bb5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.506092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.506729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.507011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.507286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] PATH_INFO: `/v3/users/831a298ff43d4d9084562dcd29ac23b6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.508560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831a298ff43d4d9084562dcd29ac23b6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.510500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.517443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aedf8-34ee-43d4-8ca7-c94d0781345a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.521567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.522407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.522800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.523187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.525084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.536908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.538441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.547416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1027/2086] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:04.548270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.549548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.562713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1432c-a40f-4052-b537-bfdf7d3e0cf5 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.567876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.568520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.568808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.569086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.569778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.571354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.581361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1028/2087] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:04.585792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.595216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb81a1-9d7e-4462-b77d-5e2d64e063e2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.599808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.600495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.600778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.601052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] PATH_INFO: `/v3/projects/7a5d1bf2926b4a10801ed218853fc306/users/d70fcd7967044022828e63a2ada153ce/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.615984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5d1bf2926b4a10801ed218853fc306, user_id=d70fcd7967044022828e63a2ada153ce, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:04.618913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29fe4a-6bc6-488f-ab94-c0b71c131286 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:04.750451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1029/2088] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:04 2022] PUT /identity/v3/projects/7a5d1bf2926b4a10801ed218853fc306/users/d70fcd7967044022828e63a2ada153ce/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:04.760167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.760485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.760763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:04.794437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] MFA Rules not processed for user `d70fcd7967044022828e63a2ada153ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:04.796363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.801703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831a298ff43d4d9084562dcd29ac23b6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.802985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831a298ff43d4d9084562dcd29ac23b6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.810594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831a298ff43d4d9084562dcd29ac23b6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.811340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831a298ff43d4d9084562dcd29ac23b6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.824336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831a298ff43d4d9084562dcd29ac23b6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:04.862992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1030/2089] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:04.874690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:04.875998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:04.924339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:04.931513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1060/2090] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:04 2022] DELETE /identity/v3/users/831a298ff43d4d9084562dcd29ac23b6 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:04.934583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbcda7-a048-4d62-8fc4-c8f1a5372346 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:04.981053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:04.981734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:04.982018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:04.982290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:05.039316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:05.040893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bcc87-5665-48bd-8d65-467b870b6b10 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:05.049728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1031/2091] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:34:05.632240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:05.633788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:05.707985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb081339-4165-4c5d-a637-c214942dec88 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:05.712683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:05.713331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:05.713614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:05.713895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] PATH_INFO: `/v3/projects/57faf360e635488c8744b7399c53802e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:05.715364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57faf360e635488c8744b7399c53802e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:05.717304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:05.837781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:05.940427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57faf360e635488c8744b7399c53802e', 'request_id': 'req-49992dae-3868-4e34-af4f-700605ea0889'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:05.948147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57faf360e635488c8744b7399c53802e', 'request_id': 'req-49992dae-3868-4e34-af4f-700605ea0889'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:06.094609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57faf360e635488c8744b7399c53802e', 'request_id': 'req-49992dae-3868-4e34-af4f-700605ea0889'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:06.107766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] The token cache is being invalidate because project 57faf360e635488c8744b7399c53802e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:06.113735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:06.113735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-49992dae-3868-4e34-af4f-700605ea0889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:06.113735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1061/2092] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:05 2022] DELETE /identity/v3/projects/57faf360e635488c8744b7399c53802e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:06.117950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.118241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.118518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.139324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:06.140718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:06.200695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1032/2093] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:06.209175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.209465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.209742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.230104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:06.231304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:06.239221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1062/2094] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:06.243426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.253105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae3bc-bad7-46b1-8e16-0411d4980143 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.259814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.262891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.262891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.262891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.262891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:06.262891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.264103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.277230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1033/2095] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:34:06.281639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.296252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006becb5-cd93-4ce2-8b97-d2a240ac122a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.297650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.298267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.298552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.298945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.299614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.301597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.322936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.324642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:06.380761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5791-91ef-4573-9355-ce07a9575237 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.384714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.387950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.387950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.387950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.387950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:06.429790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1063/2096] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:06 2022] POST /identity/v3/projects => generated 386 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:06.437074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.455478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.457422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.467616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1034/2097] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:34:06.508692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0167-e116-445b-b645-53f7b60743dc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.513432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.514047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.514327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.514597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.515262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.517482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.700016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1064/2098] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:06 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:06.703601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.717951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368951-c315-432c-ad8e-540efa96ad28 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.724061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.724796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.725827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.728740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.728740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.728740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.741089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1035/2099] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:06 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:06.745220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.755156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb99cc4-d136-4142-9e96-aac193483371 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.760590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.761202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.761483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.761760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.762457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.764057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afd8d-1595-4850-a4a3-ad213fb25555 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:06.776115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1065/2100] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:06 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:06.780484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:06.791081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb0e1-e3fc-4bb1-b909-09a298b6167d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:06.795578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:06.796241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:06.796556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:06.796837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] PATH_INFO: `/v3/projects/58fc88e5d9ab4887bccd78d4d8929878/users/b27f2b367a4b4eaebb69aff4f9bfb113/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:06.812858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fc88e5d9ab4887bccd78d4d8929878, user_id=b27f2b367a4b4eaebb69aff4f9bfb113, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:06.815637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.010321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1036/2101] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:06 2022] PUT /identity/v3/projects/58fc88e5d9ab4887bccd78d4d8929878/users/b27f2b367a4b4eaebb69aff4f9bfb113/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:07.022184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.022537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.022890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.053572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] MFA Rules not processed for user `b27f2b367a4b4eaebb69aff4f9bfb113`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:07.054791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:07.141722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1066/2102] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:07.146720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:07.215030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7743d00-dcfc-4c6e-90ef-d936c73c24c7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:07.220033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:07.220727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.221137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.221422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.222093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:07.224788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.385824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1037/2103] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:07 2022] POST /identity/v3/projects => generated 384 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:07.386964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:07.398924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4b04e-1f81-4c3b-a387-26e04c8c4456 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:07.455035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:07.455762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.456151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.456591 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.458057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:07.460071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.644285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1067/2104] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:07 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:07.648243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:07.658420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bffd4-1e7d-4799-a5d4-8f49b2fed271 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:07.663433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:07.664037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.664314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.664631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.665324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:07.666921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.677307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1038/2105] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:07 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:07.681277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:07.691139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d866b9-4985-470d-8b24-ae7f86c227f3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:07.696037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:07.696706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.696986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.697348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.697965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:07.699578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312d733-f3c9-44cd-b7b6-a264cfb8c0af admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.711078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1068/2106] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:07 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:07.715319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:07.726439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b635-9fdc-4a86-8acf-205e823f48c2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:07.743439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:07.743439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.743439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.743439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] PATH_INFO: `/v3/projects/08e3ec8140e9455db41062852ce66732/users/767b0d88e19c4ff39dc8e4f6580c7130/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.762954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e3ec8140e9455db41062852ce66732, user_id=767b0d88e19c4ff39dc8e4f6580c7130, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:07.771615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:07.833388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:07.936177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1039/2107] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:07 2022] PUT /identity/v3/projects/08e3ec8140e9455db41062852ce66732/users/767b0d88e19c4ff39dc8e4f6580c7130/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:07.949257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:07.949738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:07.950030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:07.982507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] MFA Rules not processed for user `767b0d88e19c4ff39dc8e4f6580c7130`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:07.984115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:08.052415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1069/2108] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:08.057204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:08.125437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c434f5-e7e2-44dc-a4de-98d9389c7a08 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:08.131944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:08.132897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:08.132897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:08.133231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:08.133822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:08.136376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:08.439304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1040/2109] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:08 2022] POST /identity/v3/projects => generated 384 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:08.444017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:08.454785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256aed74-17d4-439a-bba7-fbef55ccc854 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:08.512345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:08.513008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:08.513293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:08.513568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:08.514268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:08.516468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:08.758687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1070/2110] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:08 2022] POST /identity/v3/users => generated 299 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:08.762778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:08.773219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a622c6-ca6b-4615-93f1-8afed0717579 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:08.777738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:08.778338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:08.778614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:08.778950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:08.780085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:08.781786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:08.797039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1041/2111] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:08 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:08.801859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:08.812485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db5d54-7005-410b-b175-31c14a209466 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:08.819135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74852639-1273-4728-849e-331626e4f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:08.819924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74852639-1273-4728-849e-331626e4f30f admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:08.820203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74852639-1273-4728-849e-331626e4f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:08.820512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74852639-1273-4728-849e-331626e4f30f admin admin] PATH_INFO: `/v3/projects/b90bd07bdb39490d9de16babe649a58b/users/49e2b043bf3542a580f041f2f0e57136/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:08.837137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74852639-1273-4728-849e-331626e4f30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90bd07bdb39490d9de16babe649a58b, user_id=49e2b043bf3542a580f041f2f0e57136, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:08.839796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74852639-1273-4728-849e-331626e4f30f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:08.992975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1071/2112] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:08 2022] PUT /identity/v3/projects/b90bd07bdb39490d9de16babe649a58b/users/49e2b043bf3542a580f041f2f0e57136/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:08.996916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:09.007256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a950-ed07-4b71-a77c-d51a5a3bf814 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:09.064487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:09.065132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:09.065419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:09.065698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:09.066403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:09.067999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8288fc6-0b5f-4aed-88c6-75ff2c161210 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:09.079076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1042/2113] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:08 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:09.083117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74852639-1273-4728-849e-331626e4f30f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:09.094021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74852639-1273-4728-849e-331626e4f30f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:09.099021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:09.099706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:09.100075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:09.100279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] PATH_INFO: `/v3/projects/b90bd07bdb39490d9de16babe649a58b/users/49e2b043bf3542a580f041f2f0e57136/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:09.102289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90bd07bdb39490d9de16babe649a58b, user_id=49e2b043bf3542a580f041f2f0e57136, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:09.104948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:09.116094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1072/2114] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:09 2022] PUT /identity/v3/projects/b90bd07bdb39490d9de16babe649a58b/users/49e2b043bf3542a580f041f2f0e57136/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:09.126266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:09.126574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:09.126849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:09.154731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] MFA Rules not processed for user `49e2b043bf3542a580f041f2f0e57136`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:09.155952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:09.224865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1043/2115] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:09.504114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:09.505909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:09.585304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440dc3-66bb-4129-8ed5-e993823091fd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:09.589156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:09.589874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:09.590224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:09.590559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:09.649319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:09.650785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:09.660358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1073/2116] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:10.889902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:11.892768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:13.923861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:14.169365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:15.648911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:16.084055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:18.886308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:18.898300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99985c2a-f7f9-4903-85f9-0bce93e8f68c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:18.902844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:18.903474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:18.903774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:18.904114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:18.965591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:18.967068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:18.976886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1044/2117] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:19.609434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:20.323904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:21.719953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:21.729603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5857a8-5246-4f01-9b57-7225e73d63df service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:21.733756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:21.734451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:21.734729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:21.735000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:21.745614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:21.747087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9771-6b26-4d59-86c3-8669caa7f73c service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:21.757414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1074/2118] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:23.105234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:23.109208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:23.249565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2e39-8a61-44e7-80b6-ec96f00b2574 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:23.258330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:23.259205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:23.259586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:23.260038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] PATH_INFO: `/v3/users/d70fcd7967044022828e63a2ada153ce` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:23.261584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70fcd7967044022828e63a2ada153ce)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:23.263642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:23.358084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:23.916466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70fcd7967044022828e63a2ada153ce'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:23.921048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70fcd7967044022828e63a2ada153ce'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:24.278288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70fcd7967044022828e63a2ada153ce'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:24.278982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70fcd7967044022828e63a2ada153ce'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:24.287469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70fcd7967044022828e63a2ada153ce'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:24.314224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1045/2119] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:23 2022] DELETE /identity/v3/users/d70fcd7967044022828e63a2ada153ce => generated 0 bytes in 1209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:24.319114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:24.319623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:24.319726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:24.339379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:24.340615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:24.418512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1075/2120] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:24.430728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:24.432676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:24.499526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6714315-b88f-4e21-b670-22afaab43b35 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:24.505070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:24.505926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:24.506209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:24.506483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:24.521572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:24.523292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:24.539383 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1046/2121] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:24.959286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:25.414916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:25.417100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:25.431112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c8106-212a-4f76-9dae-6b4f1c1bffa3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:25.436845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:25.437760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:25.438335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:25.438836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] PATH_INFO: `/v3/projects/7a5d1bf2926b4a10801ed218853fc306` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:25.440258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5d1bf2926b4a10801ed218853fc306)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:25.442393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:25.636301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5d1bf2926b4a10801ed218853fc306', 'request_id': 'req-37892a9b-c966-4700-b31b-5cbf164b8e59'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:25.643790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5d1bf2926b4a10801ed218853fc306', 'request_id': 'req-37892a9b-c966-4700-b31b-5cbf164b8e59'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:25.809677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5d1bf2926b4a10801ed218853fc306', 'request_id': 'req-37892a9b-c966-4700-b31b-5cbf164b8e59'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:25.820924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] The token cache is being invalidate because project 7a5d1bf2926b4a10801ed218853fc306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:25.821165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:25.821822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:25.823373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1076/2122] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:25 2022] DELETE /identity/v3/projects/7a5d1bf2926b4a10801ed218853fc306 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:25.954578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:25.956145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:25.995043 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13300-ff5e-4e34-b6c0-b5dfabd82984 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:26.058209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:26.058856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:26.059137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:26.059409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] PATH_INFO: `/v3/users/db18d0211f2a4e1c8949726ded7a9bb3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:26.060698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db18d0211f2a4e1c8949726ded7a9bb3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:26.062632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:26.428727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db18d0211f2a4e1c8949726ded7a9bb3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:26.429392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db18d0211f2a4e1c8949726ded7a9bb3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:26.614602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db18d0211f2a4e1c8949726ded7a9bb3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:26.615202 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db18d0211f2a4e1c8949726ded7a9bb3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:26.626753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8df4dc0e-c242-423d-a37d-2c0aa060692a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db18d0211f2a4e1c8949726ded7a9bb3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:26.647203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1047/2123] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:25 2022] DELETE /identity/v3/users/db18d0211f2a4e1c8949726ded7a9bb3 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:27.071577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:27.071577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:27.081867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37892a9b-c966-4700-b31b-5cbf164b8e59 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:27.146933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:27.147585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:27.147866 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:27.148140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] PATH_INFO: `/v3/projects/940d1a5ec6514d18946628b99f16a96e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:27.149335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=940d1a5ec6514d18946628b99f16a96e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:27.151469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:27.261184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:27.277927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940d1a5ec6514d18946628b99f16a96e', 'request_id': 'req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:27.284342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940d1a5ec6514d18946628b99f16a96e', 'request_id': 'req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:27.511889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940d1a5ec6514d18946628b99f16a96e', 'request_id': 'req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:27.524953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] The token cache is being invalidate because project 940d1a5ec6514d18946628b99f16a96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:27.525246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:27.526844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-920aa3fb-5c3c-453e-b8cc-e2058bfc2abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:27.527395 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1077/2124] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:27 2022] DELETE /identity/v3/projects/940d1a5ec6514d18946628b99f16a96e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:27.532440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:27.532730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:27.533004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:27.554845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:27.556821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:27.617644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1048/2125] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:27.626088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:27.626375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:27.626667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:27.648565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:27.649809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:27.658435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1078/2126] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:27.662824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:27.672672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53430387-6d22-47a7-a479-f76c3febb9c0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:27.677416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:27.678031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:27.678316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:27.678596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:27.679324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:27.679630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:27.681916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:27.693628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1049/2127] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:27.697808 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:27.707121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865f6b-23fd-406c-ae89-63796ecddfcb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:27.712624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:27.713034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:27.713316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:27.713590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:27.714288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:27.716187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.183028 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1079/2128] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:27 2022] POST /identity/v3/projects => generated 390 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:28.190276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.200549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ab672-666e-4502-a22d-03aa3f3d41b3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.257736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.258358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.258635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.258904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.259593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.261518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.491237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1050/2129] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:28 2022] POST /identity/v3/users => generated 296 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:28.496677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.507347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff846-aedf-4d9d-89d5-6ca36b776b56 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.512434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.513202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.513498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.513764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.514485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.516100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.527493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1080/2130] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:28.531695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.541958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c088f-55a0-42b7-a336-daefd91bc4d7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.546628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.547308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.547510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.547789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.548524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.550113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9bc0e-e6dd-4d4b-b7a1-b9553942edff admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.561076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1051/2131] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:28.565785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.577246 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008ceaa-fee7-4a75-bbff-bd3a05053601 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.582228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.583467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.583761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.584039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] PATH_INFO: `/v3/projects/f6a8f13fe22741d6843f193a0a959e57/users/09de5ea60b9943adacc765b5ecdccb71/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.599507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a8f13fe22741d6843f193a0a959e57, user_id=09de5ea60b9943adacc765b5ecdccb71, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.602197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.697787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1081/2132] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:28 2022] PUT /identity/v3/projects/f6a8f13fe22741d6843f193a0a959e57/users/09de5ea60b9943adacc765b5ecdccb71/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:28.703115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.703608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.703868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.723330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:28.724587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:28.782151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1052/2133] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:28.792001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.795285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:28.861328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58733a6-c190-430b-b0ea-8201c6127d27 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.865766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.866381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.866660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.866934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.876662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.878109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d365d-a094-4658-89da-43fb62371e88 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:28.889471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1082/2134] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:28.967810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:28.969250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:28.980959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a273f7-3e18-4c1b-94c4-5bf7fe4606dd None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:28.985564 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:28.986188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:28.986471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:28.986747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] PATH_INFO: `/v3/users/b27f2b367a4b4eaebb69aff4f9bfb113` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:28.987905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27f2b367a4b4eaebb69aff4f9bfb113)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:28.990380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:29.069111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:29.399187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27f2b367a4b4eaebb69aff4f9bfb113'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:29.399837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27f2b367a4b4eaebb69aff4f9bfb113'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:29.472966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27f2b367a4b4eaebb69aff4f9bfb113'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:29.474090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27f2b367a4b4eaebb69aff4f9bfb113'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:29.484844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27f2b367a4b4eaebb69aff4f9bfb113'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:29.503771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1053/2135] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:28 2022] DELETE /identity/v3/users/b27f2b367a4b4eaebb69aff4f9bfb113 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:29.508449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:29.508946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:29.509033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:29.528616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:29.530120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:29.592331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1083/2136] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:29.602453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:29.604040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:29.665487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ac70-3323-45a8-a9f9-f63bcafe3c6f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:29.669163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:29.669798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:29.670086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:29.670364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:29.681301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:29.682975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:29.684291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:29.699647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1054/2137] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:30.183183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:30.184906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:30.205014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226b93d-1a66-4696-be67-ab02c408bb39 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:30.222269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:30.222269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:30.222269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:30.222269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] PATH_INFO: `/v3/projects/58fc88e5d9ab4887bccd78d4d8929878` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:30.222647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fc88e5d9ab4887bccd78d4d8929878)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:30.224985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:30.349525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fc88e5d9ab4887bccd78d4d8929878', 'request_id': 'req-958c9dc1-27c9-4464-becb-f6fae19f255d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:30.356021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fc88e5d9ab4887bccd78d4d8929878', 'request_id': 'req-958c9dc1-27c9-4464-becb-f6fae19f255d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:30.523886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fc88e5d9ab4887bccd78d4d8929878', 'request_id': 'req-958c9dc1-27c9-4464-becb-f6fae19f255d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:30.537128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] The token cache is being invalidate because project 58fc88e5d9ab4887bccd78d4d8929878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:30.537431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:30.538092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:30.539676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1084/2138] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:30 2022] DELETE /identity/v3/projects/58fc88e5d9ab4887bccd78d4d8929878 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:30.543963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:30.555726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b581-8e70-442d-9d30-6bcb85ba8317 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:30.618960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:30.619619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:30.619908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:30.620209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] PATH_INFO: `/v3/users/767b0d88e19c4ff39dc8e4f6580c7130` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:30.621415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=767b0d88e19c4ff39dc8e4f6580c7130)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:30.623355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:31.071877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767b0d88e19c4ff39dc8e4f6580c7130'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:31.072474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767b0d88e19c4ff39dc8e4f6580c7130'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:31.248703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767b0d88e19c4ff39dc8e4f6580c7130'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:31.249519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767b0d88e19c4ff39dc8e4f6580c7130'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:31.259741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767b0d88e19c4ff39dc8e4f6580c7130'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:31.288476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1055/2139] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:30 2022] DELETE /identity/v3/users/767b0d88e19c4ff39dc8e4f6580c7130 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:31.632879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:31.635388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:31.650947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c9dc1-27c9-4464-becb-f6fae19f255d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:31.731300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:31.732788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:31.733910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:31.734503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] PATH_INFO: `/v3/users/b25da672e758453597c14eeff3184947` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:31.735992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25da672e758453597c14eeff3184947)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:31.738190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:31.887633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:31.890233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:31.935488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dead9a-535e-4e00-ad9f-21f1c5068d26 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:31.938598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:31.939421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:31.939805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:31.940175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] PATH_INFO: `/v3/projects/08e3ec8140e9455db41062852ce66732` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:31.943279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e3ec8140e9455db41062852ce66732)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:31.947541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:32.081177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25da672e758453597c14eeff3184947'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.081834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25da672e758453597c14eeff3184947'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.089747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25da672e758453597c14eeff3184947'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.094128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25da672e758453597c14eeff3184947'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.105442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ecd0608-3a8a-4328-a3e0-59177d9fb5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25da672e758453597c14eeff3184947'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.108945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e3ec8140e9455db41062852ce66732', 'request_id': 'req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.116331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e3ec8140e9455db41062852ce66732', 'request_id': 'req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.122234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e3ec8140e9455db41062852ce66732', 'request_id': 'req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.221312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1085/2140] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:31 2022] DELETE /identity/v3/users/b25da672e758453597c14eeff3184947 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:32.226580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:32.226933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:32.227736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:32.249490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:32.250941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:32.330598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1086/2141] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:32.344571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:32.346853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:32.363346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] The token cache is being invalidate because project 08e3ec8140e9455db41062852ce66732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:32.364737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.365611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.368660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1056/2142] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:31 2022] DELETE /identity/v3/projects/08e3ec8140e9455db41062852ce66732 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:32.379899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:32.388208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:32.425728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84070b9-cf94-487d-88eb-2f5700743716 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:32.432102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:32.432767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:32.433205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:32.433484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:32.435616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:32.450639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:32.452501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:32.470428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1087/2143] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:34:32.476759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1500de2-5bb4-43b4-82cb-ae0a621b2a88 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:32.482538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:32.483205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:32.483487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:32.483762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] PATH_INFO: `/v3/users/49e2b043bf3542a580f041f2f0e57136` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:32.485213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e2b043bf3542a580f041f2f0e57136)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:32.487578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:32.899930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e2b043bf3542a580f041f2f0e57136'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.903046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e2b043bf3542a580f041f2f0e57136'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:32.929742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:33.112058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e2b043bf3542a580f041f2f0e57136'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.112776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e2b043bf3542a580f041f2f0e57136'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.124528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-879ec36b-fc22-4a43-b3dc-81528bfcc8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e2b043bf3542a580f041f2f0e57136'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.149338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1057/2144] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:32 2022] DELETE /identity/v3/users/49e2b043bf3542a580f041f2f0e57136 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:33.343678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:33.346755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:33.366525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e90a3-09c1-438f-9c65-bed401995670 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:33.454364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:33.455108 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:33.455756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:33.456234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] PATH_INFO: `/v3/projects/49f71964c1af45649fbe7154f30f298c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:33.459506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f71964c1af45649fbe7154f30f298c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:33.463682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:33.684226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f71964c1af45649fbe7154f30f298c', 'request_id': 'req-a08ac40f-bae7-4ae7-9190-ba5e419344fe'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.698378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f71964c1af45649fbe7154f30f298c', 'request_id': 'req-a08ac40f-bae7-4ae7-9190-ba5e419344fe'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.883861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f71964c1af45649fbe7154f30f298c', 'request_id': 'req-a08ac40f-bae7-4ae7-9190-ba5e419344fe'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.884441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:33.899928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] The token cache is being invalidate because project 49f71964c1af45649fbe7154f30f298c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:33.900424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.903521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a08ac40f-bae7-4ae7-9190-ba5e419344fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:33.905515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1088/2145] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:33 2022] DELETE /identity/v3/projects/49f71964c1af45649fbe7154f30f298c => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:33.910107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:33.910658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:33.910754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:33.937330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:33.938658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.032456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1058/2146] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:34.040122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.040638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.040638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.082991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:34.084339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.096275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1089/2147] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:34.099180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.109282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f0845-682f-4ca0-8da3-da5ad08dbbe1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.117017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.117853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.118311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.118746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.119772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:34.120245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.122731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.140699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1059/2148] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:34.145470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.156660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc881dac-ac1c-4cdb-b603-5f45091007bb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.161110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.161849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.162235 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.162608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.163587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.165751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.188745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.190594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.220054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfe03c-2d44-4016-b872-751a33ae1f31 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.226233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.227154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.227552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.227936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] PATH_INFO: `/v3/projects/b90bd07bdb39490d9de16babe649a58b` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.230024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90bd07bdb39490d9de16babe649a58b)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.232803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.297739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1090/2149] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/projects => generated 390 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:34.298530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.370942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd418-8e92-4f58-b24a-8c1a7527981e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.375958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.376608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.377172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.377367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.378036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.380632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.443986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90bd07bdb39490d9de16babe649a58b', 'request_id': 'req-a54c2364-f713-4b02-b010-3948c746af24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:34.451262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90bd07bdb39490d9de16babe649a58b', 'request_id': 'req-a54c2364-f713-4b02-b010-3948c746af24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:34.458766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90bd07bdb39490d9de16babe649a58b', 'request_id': 'req-a54c2364-f713-4b02-b010-3948c746af24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:34.577579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1091/2150] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/users => generated 296 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:34:34.581789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.586953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] The token cache is being invalidate because project b90bd07bdb39490d9de16babe649a58b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:34.587587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:34.588503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-a54c2364-f713-4b02-b010-3948c746af24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:34.593973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1060/2151] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:34 2022] DELETE /identity/v3/projects/b90bd07bdb39490d9de16babe649a58b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:34.595546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b04a-3cfe-4bad-a401-d908909f8651 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.596029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b04a-3cfe-4bad-a401-d908909f8651 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.596553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b04a-3cfe-4bad-a401-d908909f8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.600692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8081ff-de91-4761-b3ca-a49ec55a2247 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.618847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2b89b04a-3cfe-4bad-a401-d908909f8651 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:34.620902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2b89b04a-3cfe-4bad-a401-d908909f8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.681663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.682481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.682768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.683074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.683748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.685718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.700242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1092/2152] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:34 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:34.701365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.703486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.713914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1061/2153] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:34.723317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.724535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.724988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.749574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:34.751253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:34.760901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1062/2154] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:34.775284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e4d6-419e-4cdd-9bbd-031331afdbeb admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.776644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.778222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.781983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.782578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.783053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.784552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.789162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.792020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e49d-6d47-49ad-8fde-d299c213987f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.797668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.798344 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.798741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.799228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.799964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:34.800522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.802171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.810492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1093/2155] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:34 2022] GET /identity/v3/roles => generated 1427 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:34.818672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.828514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1063/2156] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:34:34.831806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:34.836553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833edd-eb58-46a5-b5f5-9d83badd447e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.842179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.846684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.847167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.847626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] PATH_INFO: `/v3/projects/9fd73ee0b6b04b499f9f89c2b7535021/users/bdbc9bc51c8240e28aadf937a2002167/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.850959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf11ad-8873-4039-96af-a507fe0108d4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:34.866610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:34.867696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:34.868276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:34.868808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:34.870665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.872499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd73ee0b6b04b499f9f89c2b7535021, user_id=bdbc9bc51c8240e28aadf937a2002167, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:34.873586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:34.875187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70fed4-2171-4814-8a12-1216c5eca641 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.014952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1094/2157] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:34 2022] PUT /identity/v3/projects/9fd73ee0b6b04b499f9f89c2b7535021/users/bdbc9bc51c8240e28aadf937a2002167/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:35.026376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.026611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.026892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.054266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1064/2158] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:34 2022] POST /identity/v3/projects => generated 386 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:35.055949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.056824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] MFA Rules not processed for user `bdbc9bc51c8240e28aadf937a2002167`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:35.058129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:35.128237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bca3d-f1b1-4570-a36c-988c82cf3807 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.135158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.136225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.136683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.137296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.138565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.141560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.143786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1095/2159] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:35.157960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.159473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:35.221939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd051-ecbe-4ea2-95d4-705e5832204e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.227340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.227938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.228572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.228691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.240945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.242396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.252561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1096/2160] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:35.273840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1065/2161] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:35 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:35.278576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.295660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef0a1e-b3ef-4ed9-ace5-000a757128c7 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.305211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.305870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.306496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.309806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.309806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.311205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.326316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1097/2162] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:35 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:35.330207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.343351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72020354-43f3-430f-8ffd-845d9905cc5a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.349530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.350634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.351111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.351578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.352530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.354440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d56b-8884-47d8-90df-92fe4b494da6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.370987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1066/2163] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:35 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:35.373178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.384788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59d89-2931-419d-9637-f5f73254da2d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.392055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.392962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.393442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.393883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] PATH_INFO: `/v3/projects/1e96766033024eab80c1073d4cc8b341/users/33f1fa525763425eb87e4ecf550573e4/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.411568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e96766033024eab80c1073d4cc8b341, user_id=33f1fa525763425eb87e4ecf550573e4, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.414923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.529933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1098/2164] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:35 2022] PUT /identity/v3/projects/1e96766033024eab80c1073d4cc8b341/users/33f1fa525763425eb87e4ecf550573e4/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:35.540966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b468c-4a40-4362-8687-2ec03fdcad51 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.541264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b468c-4a40-4362-8687-2ec03fdcad51 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.541568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b468c-4a40-4362-8687-2ec03fdcad51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.571616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-222b468c-4a40-4362-8687-2ec03fdcad51 None None] MFA Rules not processed for user `33f1fa525763425eb87e4ecf550573e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:35.572865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-222b468c-4a40-4362-8687-2ec03fdcad51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:35.665197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1067/2165] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:35.704836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:35.706894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:35.770090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c95683-0a72-45b0-8ca8-fe644023323b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:35.774066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:35.774804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:35.774967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:35.775241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:35.788762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:35.790307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:35.800475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1099/2166] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:36.157640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:36.158142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:36.169012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:36.189146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] MFA Rules not processed for user `09de5ea60b9943adacc765b5ecdccb71`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:36.190469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:36.287647 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:36.300259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1068/2167] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:36.319016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:36.324229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:36.409501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daba7-4639-4ef8-a2bf-cb37f03837ff service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:36.416567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:36.417257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:36.420918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:36.420918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:36.427962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:36.429823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:36.440115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1100/2168] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:38.412458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:38.415990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:38.568237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f2015-a442-4ef3-b89a-a334ad260d33 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:38.573597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:38.575291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:38.575783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:38.576064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] PATH_INFO: `/v3/users/15d0b7566d684afa871672aabb17c2cc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:38.577333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d0b7566d684afa871672aabb17c2cc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:38.579275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:38.955303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d0b7566d684afa871672aabb17c2cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:38.956425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d0b7566d684afa871672aabb17c2cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:39.106579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d0b7566d684afa871672aabb17c2cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:39.107322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d0b7566d684afa871672aabb17c2cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:39.119352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d0b7566d684afa871672aabb17c2cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:39.145992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1069/2169] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:38 2022] DELETE /identity/v3/users/15d0b7566d684afa871672aabb17c2cc => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:39.226751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:39.504166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:39.516374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137c99a-c8fa-4c6d-9a78-7d1cc92fd77a service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:39.613449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:39.615283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:39.616368 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:39.616905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] PATH_INFO: `/v3/projects/686d573bd8724d0a92250816f3432d2e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:39.622901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686d573bd8724d0a92250816f3432d2e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:39.625122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:39.807949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686d573bd8724d0a92250816f3432d2e', 'request_id': 'req-7f307f24-6592-4743-833a-7986345d5e81'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:39.814265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686d573bd8724d0a92250816f3432d2e', 'request_id': 'req-7f307f24-6592-4743-833a-7986345d5e81'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.031724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686d573bd8724d0a92250816f3432d2e', 'request_id': 'req-7f307f24-6592-4743-833a-7986345d5e81'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.046901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] The token cache is being invalidate because project 686d573bd8724d0a92250816f3432d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:40.047322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.048077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7f307f24-6592-4743-833a-7986345d5e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.050112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1101/2170] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:39 2022] DELETE /identity/v3/projects/686d573bd8724d0a92250816f3432d2e => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:40.054750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:40.057551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:40.071710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e72869-cbc9-42a5-be66-45c8a5d5db5a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:40.138713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:40.139636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:40.139983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:40.140324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] PATH_INFO: `/v3/users/c124f87901294a4b992adf8a2691a423` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:40.141617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c124f87901294a4b992adf8a2691a423)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:40.143893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:40.497525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c124f87901294a4b992adf8a2691a423'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.498182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c124f87901294a4b992adf8a2691a423'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.631880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c124f87901294a4b992adf8a2691a423'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.632581 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c124f87901294a4b992adf8a2691a423'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.640070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c124f87901294a4b992adf8a2691a423'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:40.657780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1070/2171] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:40 2022] DELETE /identity/v3/users/c124f87901294a4b992adf8a2691a423 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:40.751976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245a82-f202-48ee-9f45-0dd4634613f8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:40.757054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245a82-f202-48ee-9f45-0dd4634613f8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:40.757536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245a82-f202-48ee-9f45-0dd4634613f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:40.781739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e0245a82-f202-48ee-9f45-0dd4634613f8 None None] MFA Rules not processed for user `09de5ea60b9943adacc765b5ecdccb71`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:40.782975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0245a82-f202-48ee-9f45-0dd4634613f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:40.872629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1102/2172] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:41.009858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:41.011672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:41.087738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f479142-2181-4768-abd1-642c3cc4b2c0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:41.092715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:41.093479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.093809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.094081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] PATH_INFO: `/v3/projects/a0dd5a7a647a465b852cad5b51cbde56` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.095249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0dd5a7a647a465b852cad5b51cbde56)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:41.097212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:41.243330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0dd5a7a647a465b852cad5b51cbde56', 'request_id': 'req-12dd39f5-26e8-46ea-88b1-11d7832ae812'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:41.249568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0dd5a7a647a465b852cad5b51cbde56', 'request_id': 'req-12dd39f5-26e8-46ea-88b1-11d7832ae812'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:41.255895 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0dd5a7a647a465b852cad5b51cbde56', 'request_id': 'req-12dd39f5-26e8-46ea-88b1-11d7832ae812'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:41.357590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] The token cache is being invalidate because project a0dd5a7a647a465b852cad5b51cbde56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:41.357879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:41.358545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-12dd39f5-26e8-46ea-88b1-11d7832ae812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:41.360071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1071/2173] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:41 2022] DELETE /identity/v3/projects/a0dd5a7a647a465b852cad5b51cbde56 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:41.365515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.365799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.366075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.386452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:41.387717 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:41.453217 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1103/2174] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:41.461388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.461847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.461955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.482545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:41.483754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:41.491768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1072/2175] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:41.495964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:41.506071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d012e-0220-46d8-97bc-e85c3fc5e11d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:41.511068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:41.511736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.512019 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.512297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.513119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:41.513463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:41.515064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:41.528027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1104/2176] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:41.532336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:41.544764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6674fc-137e-4893-ae32-0d4e006b6d2c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:41.549565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:41.550210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.550493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.550787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.551552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:41.553497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:41.748968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1073/2177] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:41 2022] POST /identity/v3/projects => generated 386 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:41.752051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:41.762482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d40b-5811-4f1d-aa13-6746b6bdf4ac admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:41.821467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:41.822089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:41.822372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:41.822649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:41.823335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:41.825389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:42.026159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1105/2178] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:41 2022] POST /identity/v3/users => generated 294 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:42.030208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:42.041026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d323f6-85a6-4bbf-99c1-51abbdf95c9e admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:42.045568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:42.046208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.046487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.046760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.047458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:42.049083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:42.060226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1074/2179] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:42 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:42.064206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:42.077565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31db5d-e9f8-45d6-9502-4e8f7aee3ab6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:42.082320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:42.083209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.083489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.083764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.084489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:42.086141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4355-7763-4ba7-ad42-dfc905788b55 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:42.096998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1106/2180] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:42 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:34:42.100899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:42.116873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec9752-0731-42e9-bf98-8942897d882f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:42.121827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:42.122503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.122788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.123066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] PATH_INFO: `/v3/projects/1765cb727c78452f9a355130b2ea0c34/users/040b7593393d4ed1a5fdbbd6185bde62/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.138490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1765cb727c78452f9a355130b2ea0c34, user_id=040b7593393d4ed1a5fdbbd6185bde62, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:42.141528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:42.318267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1075/2181] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:42 2022] PUT /identity/v3/projects/1765cb727c78452f9a355130b2ea0c34/users/040b7593393d4ed1a5fdbbd6185bde62/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:42.386202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.386491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.386771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.415068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] MFA Rules not processed for user `040b7593393d4ed1a5fdbbd6185bde62`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:42.416296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:42.480869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1107/2182] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:42.498498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:42.500191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:42.572763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80184d94-9669-4e93-8842-918cbbd5a0a5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:42.576621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:42.577264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.577545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.577837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.585791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:42.586347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:42.587773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741e58-ae1c-4b86-b82a-a2cb75f87e87 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:42.596578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1076/2183] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:42.616313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:42.677190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c06f92-3f8c-4359-922c-4e9f0b6097bc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:42.681735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:42.682367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:42.682651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:42.682923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] PATH_INFO: `/v3/users/040b7593393d4ed1a5fdbbd6185bde62` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:42.684082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=040b7593393d4ed1a5fdbbd6185bde62)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:42.686029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:43.044780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040b7593393d4ed1a5fdbbd6185bde62'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:43.045465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040b7593393d4ed1a5fdbbd6185bde62'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:43.052068 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040b7593393d4ed1a5fdbbd6185bde62'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:43.052706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040b7593393d4ed1a5fdbbd6185bde62'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:43.064491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040b7593393d4ed1a5fdbbd6185bde62'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:43.181330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1108/2184] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:42 2022] DELETE /identity/v3/users/040b7593393d4ed1a5fdbbd6185bde62 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:43.185871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:43.186273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:43.186451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:43.206960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:43.208980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:43.268801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1077/2185] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:43.278128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:43.279587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:43.339883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527169c6-92a7-4c9f-82f6-b610ddf85dcb admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:43.343900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:43.344567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:43.344850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:43.345127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:43.354571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:43.356016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dddcbf-17a0-47e4-b749-51c24d43c7f3 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:43.365913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1109/2186] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:43.902815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:43.904323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:43.917303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd310193-fcd0-4eee-983b-675077157c04 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:43.927984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:43.928571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:43.928866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:43.929140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] PATH_INFO: `/v3/projects/1765cb727c78452f9a355130b2ea0c34` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:43.930473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1765cb727c78452f9a355130b2ea0c34)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:43.932337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:44.429722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1765cb727c78452f9a355130b2ea0c34', 'request_id': 'req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:44.435985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1765cb727c78452f9a355130b2ea0c34', 'request_id': 'req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:44.441705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1765cb727c78452f9a355130b2ea0c34', 'request_id': 'req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:44.677947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] The token cache is being invalidate because project 1765cb727c78452f9a355130b2ea0c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:44.678491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:44.684476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5fa1475d-acb1-4e3d-b547-e7ffa5852b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:44.684476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1078/2187] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:43 2022] DELETE /identity/v3/projects/1765cb727c78452f9a355130b2ea0c34 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:44.686102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:44.686544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:44.686683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:44.705998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:44.707199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:44.765390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1110/2188] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:44.773316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:44.773608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:44.773889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:44.793663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:44.794868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:44.803424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1079/2189] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:44.807599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:44.817066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b374eed-d1e3-4494-b12c-2225a2a43df7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:44.821632 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:44.822231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:44.822509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:44.822782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:44.823465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:44.823768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:44.825444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:44.836612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1111/2190] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:44.840760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:44.850363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f097-b5d1-49f5-a0a4-7fcb7ede4987 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:44.855111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:44.855720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:44.856002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:44.856277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:44.856974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:44.858932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:45.255384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:45.257181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:45.270184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6164519-b875-4dad-93e2-52f3c093f249 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:45.275301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:45.275931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:45.276219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:45.276522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] PATH_INFO: `/v3/users/33f1fa525763425eb87e4ecf550573e4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:45.277707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f1fa525763425eb87e4ecf550573e4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:45.279611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:45.285364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1080/2191] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:44 2022] POST /identity/v3/projects => generated 378 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:45.290038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:45.349293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126a13-1197-4c77-8428-05da24ea6e76 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:45.354551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:45.355174 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:45.355454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:45.355755 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:45.356500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:45.358425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:45.443947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:45.599358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1081/2192] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:45 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:34:45.603834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:45.607467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f1fa525763425eb87e4ecf550573e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:45.608137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f1fa525763425eb87e4ecf550573e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:45.613901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a7759-dbc2-440c-8b9a-562dd84e1b92 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:45.615979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f1fa525763425eb87e4ecf550573e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:45.616694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f1fa525763425eb87e4ecf550573e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:45.619053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:45.619672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:45.619957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:45.620233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:45.620976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:45.622575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:45.628816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb446dcb-1ecb-4398-b40e-0f380ea7eb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f1fa525763425eb87e4ecf550573e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:45.639341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1082/2193] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:45 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:45.643245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:45.645083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:45.659623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed6e73-d1e4-4021-b987-8c4fd5a56e4b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:45.664648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:45.665262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:45.665541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:45.665814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:45.666506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:45.668138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:45.679234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1083/2194] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:45 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:45.683231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:45.693808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8945-31ae-4bad-857a-5d52dd95caf9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:45.698572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:45.699218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:45.699530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:45.699808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] PATH_INFO: `/v3/projects/f3d6aac1edaf4aebad2c00577e26e4f8/users/c213e18f6fac4354bae2b6537d355eaa/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:45.714823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d6aac1edaf4aebad2c00577e26e4f8, user_id=c213e18f6fac4354bae2b6537d355eaa, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:45.717587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758bd1b-6679-4504-a5b9-f4939259a769 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:46.141253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1084/2195] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:45 2022] PUT /identity/v3/projects/f3d6aac1edaf4aebad2c00577e26e4f8/users/c213e18f6fac4354bae2b6537d355eaa/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:46.143057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1112/2196] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:34:45 2022] DELETE /identity/v3/users/33f1fa525763425eb87e4ecf550573e4 => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:34:46.147956 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:46.148353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:46.148741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:46.152700 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:46.152998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:46.153276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:46.169289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:46.196858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:46.197032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] MFA Rules not processed for user `c213e18f6fac4354bae2b6537d355eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:46.197032 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:46.232426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1085/2197] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:46.244404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:46.244404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:46.254194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1113/2198] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:46.265798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:46.267450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:46.371925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b21f77-75d3-4ec4-8ad0-3e04362a01d9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:46.375568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:46.376180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:46.376498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:46.376780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:46.386656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:46.388116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:46.400944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1086/2199] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:34:46.427659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89b603-c8ee-4b53-9473-ead35c9633dc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:46.431436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:46.432054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:46.437071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:46.437071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:46.441653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:46.443096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df6af7-04b0-4bf0-aaf3-cec55bb557ed service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:46.453196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1114/2200] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:34:46.994148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:46.995638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:47.009447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7573-8d67-4aba-8cea-09915787123a service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.014794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.017761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.017761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.017761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] PATH_INFO: `/v3/projects/1e96766033024eab80c1073d4cc8b341` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.017761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e96766033024eab80c1073d4cc8b341)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.019384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.186816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e96766033024eab80c1073d4cc8b341', 'request_id': 'req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:47.193056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e96766033024eab80c1073d4cc8b341', 'request_id': 'req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:47.198592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e96766033024eab80c1073d4cc8b341', 'request_id': 'req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:47.408979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] The token cache is being invalidate because project 1e96766033024eab80c1073d4cc8b341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:34:47.409276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:47.409934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-69814e01-1b0b-4fdd-9bc1-3159aeda6adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:34:47.411532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1087/2201] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:34:46 2022] DELETE /identity/v3/projects/1e96766033024eab80c1073d4cc8b341 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:47.417048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.417524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.417618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.436853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:47.438047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:47.498184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1115/2202] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:47.506381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53011b2-e631-4350-a97f-85376e356718 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.506739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53011b2-e631-4350-a97f-85376e356718 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.507080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53011b2-e631-4350-a97f-85376e356718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.528373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a53011b2-e631-4350-a97f-85376e356718 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:47.529623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a53011b2-e631-4350-a97f-85376e356718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:47.538243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1088/2203] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:34:47.542585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:47.552399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4510ab8-c9a0-4601-bbac-1fac40bd4c88 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.558405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.559033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.559301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.560174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.560487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:34:47.560781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.562367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.576435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1116/2204] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:34:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:34:47.584706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53011b2-e631-4350-a97f-85376e356718 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:47.594196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53011b2-e631-4350-a97f-85376e356718 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.598924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.599537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.600416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.600692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.601373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.603288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.722100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1089/2205] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:34:47 2022] POST /identity/v3/projects => generated 376 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:34:47.723447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:47.734177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab02ca-7a35-4fc3-b401-cc55ff77c6ef admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.789722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.790354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.790637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.790912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.791573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.793534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.920772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1117/2206] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:34:47 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:34:47.922240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:47.932812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbba112-69c6-4782-a5c1-beab1a87bc1b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.937454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.938060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.938339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.938695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.939393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.941044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.952412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1090/2207] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:47 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:47.956536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:47.967151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe80bc-5bd1-45e5-8916-3593b63de29a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:47.972112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:47.972762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:47.973042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:47.973316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:47.974001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:47.975701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd249f5b-9170-457e-b0a1-010beb5c0f05 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:47.988736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1118/2208] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:34:47 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:34:47.996308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:48.010305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d21387-e61a-4a32-9308-2b164ea9d7ae admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:48.016830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:48.017775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:48.018054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:48.018398 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] PATH_INFO: `/v3/projects/fcbae11ef93f4951a3bc244ef9fd7a05/users/75c919b5c8624a2ebad5bff722a3ff11/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:48.034625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbae11ef93f4951a3bc244ef9fd7a05, user_id=75c919b5c8624a2ebad5bff722a3ff11, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:48.037468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:48.163731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1091/2209] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:34:47 2022] PUT /identity/v3/projects/fcbae11ef93f4951a3bc244ef9fd7a05/users/75c919b5c8624a2ebad5bff722a3ff11/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:34:48.174815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bfd1-98fe-4195-8198-e6d9490be08c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:48.175107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bfd1-98fe-4195-8198-e6d9490be08c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:48.175383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bfd1-98fe-4195-8198-e6d9490be08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:48.202226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-18f8bfd1-98fe-4195-8198-e6d9490be08c None None] MFA Rules not processed for user `75c919b5c8624a2ebad5bff722a3ff11`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:48.203449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-18f8bfd1-98fe-4195-8198-e6d9490be08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:48.238595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:48.270947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1119/2210] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:48.290605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:34:48.290605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:48.341913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7abd7c-2f43-4f63-95a0-92c1ed69c94f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:34:48.345697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:34:48.346331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:48.346606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:48.346881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:48.356379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:34:48.357831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:34:48.367077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1092/2211] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:34:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:34:48.421228 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:50.506142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:51.165589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:52.803808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:52.893179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974b8a-052e-4250-89e5-1354422a7a93 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:34:52.893686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974b8a-052e-4250-89e5-1354422a7a93 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:34:52.893856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b974b8a-052e-4250-89e5-1354422a7a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:34:52.923330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3b974b8a-052e-4250-89e5-1354422a7a93 None None] MFA Rules not processed for user `75c919b5c8624a2ebad5bff722a3ff11`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:34:52.925100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b974b8a-052e-4250-89e5-1354422a7a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:34:52.939397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1120/2212] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:34:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:34:53.540647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:54.386031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:56.746710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:56.986734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:58.311917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:58.449505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:34:59.906355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:00.569575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:01.193769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:02.592271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:03.455544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:04.757434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:06.286470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:06.288629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:06.353219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84e577-3773-454e-b2b3-bcca4b0d324c service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:06.357762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:06.358390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:06.358671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:06.358944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] PATH_INFO: `/v3/users/bdbc9bc51c8240e28aadf937a2002167` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:06.360101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbc9bc51c8240e28aadf937a2002167)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:06.362092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:06.454718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:06.645475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbc9bc51c8240e28aadf937a2002167'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:06.646134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbc9bc51c8240e28aadf937a2002167'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:06.768652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbc9bc51c8240e28aadf937a2002167'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:06.772230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbc9bc51c8240e28aadf937a2002167'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:06.777708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbc9bc51c8240e28aadf937a2002167'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:06.794924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1093/2213] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:35:06 2022] DELETE /identity/v3/users/bdbc9bc51c8240e28aadf937a2002167 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:06.799540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:06.800017 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:06.800110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:06.818423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:06.820133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:06.875567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1121/2214] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:06.885848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:06.887261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:06.943954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675ada-0706-41fa-be9a-4fe0f32a05ea admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:06.948471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:06.949082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:06.949366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:06.949664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:06.958597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:06.960569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187520ad-935f-4151-b8e9-a31a27dd1e70 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:06.969800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1094/2215] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:35:07.202913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:07.565626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:07.567194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:07.580629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5f930-70fb-4088-979b-cd079fc9d12c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:07.586529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:07.587173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:07.587458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:07.587733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] PATH_INFO: `/v3/projects/9fd73ee0b6b04b499f9f89c2b7535021` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:07.589014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd73ee0b6b04b499f9f89c2b7535021)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:07.590927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:07.747738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd73ee0b6b04b499f9f89c2b7535021', 'request_id': 'req-791d447d-ff14-42d7-870a-16a2f270e602'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:07.754298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd73ee0b6b04b499f9f89c2b7535021', 'request_id': 'req-791d447d-ff14-42d7-870a-16a2f270e602'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:07.912310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd73ee0b6b04b499f9f89c2b7535021', 'request_id': 'req-791d447d-ff14-42d7-870a-16a2f270e602'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:07.924485 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] The token cache is being invalidate because project 9fd73ee0b6b04b499f9f89c2b7535021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:35:07.924811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:07.925468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-791d447d-ff14-42d7-870a-16a2f270e602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:07.927115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1122/2216] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:35:07 2022] DELETE /identity/v3/projects/9fd73ee0b6b04b499f9f89c2b7535021 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:07.933056 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:07.933480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:07.933642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:07.953354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:07.954538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:08.013522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1095/2217] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:08.021987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.022318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.022602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.041832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:08.043084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:08.050984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1123/2218] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:35:08.055679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.066461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0462cb1-c36a-44ad-bb35-ab35d631a09e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.071713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.079313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.087649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1096/2219] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:35:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:35:08.091987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.101981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ec80c-45ac-4ab0-b64b-b51e3dd51076 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.107381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.107640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.107922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.108198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.108909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.110853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.249389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1124/2220] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:35:08 2022] POST /identity/v3/projects => generated 372 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:35:08.250541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.261959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44e2b-8d8b-4826-8c60-6f3a03d8ee91 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.327823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.328666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.328950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.331660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.331660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.331947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.524090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1097/2221] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:35:08 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:35:08.528166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.540010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63212124-1d8e-43f3-9e0d-214287024be7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.544661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.545293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.545571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.545844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.546541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.548170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.559305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1125/2222] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:35:08 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:35:08.563567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.573977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb7a7b-4249-464c-b263-f2068de1fdf4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.578811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.579427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.579708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.579980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.580712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.582308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5022bd-d294-4052-a961-5bbfcc62fa4e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.593923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1098/2223] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:35:08 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:35:08.598309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.614917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b4f1a-f335-4160-aced-51a8ccf506fc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.614917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.615117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.615339 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.615612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] PATH_INFO: `/v3/projects/22e3bf7bb6ef4ca8bb2353bdfe4e90a3/users/3b089f2cc22449329110fd3cfafecd53/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.630512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e3bf7bb6ef4ca8bb2353bdfe4e90a3, user_id=3b089f2cc22449329110fd3cfafecd53, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.633411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.756408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1126/2224] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:35:08 2022] PUT /identity/v3/projects/22e3bf7bb6ef4ca8bb2353bdfe4e90a3/users/3b089f2cc22449329110fd3cfafecd53/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:08.759949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818aa6-3f2e-41ee-9f91-f7394ffd51e7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.760604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818aa6-3f2e-41ee-9f91-f7394ffd51e7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.760604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818aa6-3f2e-41ee-9f91-f7394ffd51e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.780379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0f818aa6-3f2e-41ee-9f91-f7394ffd51e7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:08.781644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0f818aa6-3f2e-41ee-9f91-f7394ffd51e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:08.846717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1099/2225] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:08.857094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:08.858924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:08.917968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5fb2d-8fc3-42c9-9bbb-4d138dba5633 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:08.921903 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:08.922529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:08.922813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:08.923089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:08.933767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:08.935208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:08.948421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1127/2226] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:35:11.221452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:12.198893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:13.135173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:15.056358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:16.835786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d822d-ac97-4660-b0f4-118ff73d01eb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:16.836255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d822d-ac97-4660-b0f4-118ff73d01eb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:16.836349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d822d-ac97-4660-b0f4-118ff73d01eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:16.871942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1e6d822d-ac97-4660-b0f4-118ff73d01eb None None] MFA Rules not processed for user `3b089f2cc22449329110fd3cfafecd53`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:16.872656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e6d822d-ac97-4660-b0f4-118ff73d01eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:16.980475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1100/2227] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:16.984722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:16.996084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:16.996609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:16.996992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:17.003875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a8922-e435-4582-b740-3251969d71d7 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:17.010523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:17.011990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:17.012465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:17.012922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:17.014587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:17.014800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:17.016893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:17.031018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] MFA Rules not processed for user `07e70b1c968443eab469777283b8fb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:17.034251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:17.098978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1101/2228] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:17.216922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1128/2229] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:35:16 2022] POST /identity/v3/projects => generated 374 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:35:17.218110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:17.230047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97152f-905d-4797-94d4-587cac436aeb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:17.354439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:17.355163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:17.356747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:17.357113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:17.358721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:17.363771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:17.557736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1102/2230] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:35:17 2022] POST /identity/v3/users => generated 294 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:35:17.558948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:17.581022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aadda-bdd4-4135-9e33-5364841305d2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:17.593850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:17.594787 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:17.595071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:17.595345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:17.596223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:17.598037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:17.617551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1129/2231] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:35:17 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:35:17.622086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:17.660705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70843b-1ec5-404a-9d9a-34146f9872fc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:17.665891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:17.673122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:17.673122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:17.673122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] PATH_INFO: `/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14/users/8573cfc4ecda47ab8480769977aeac07/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:17.690909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ae6c7e6f846dba2b27407c6a30d14, user_id=8573cfc4ecda47ab8480769977aeac07, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:17.693819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:17.884116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1103/2232] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:35:17 2022] PUT /identity/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14/users/8573cfc4ecda47ab8480769977aeac07/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:17.909616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:17.920797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606b9a-4c23-4148-8f2e-eb6e3cb1fa1d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:18.000290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:18.001222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:18.001577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:18.001925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:18.002713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:18.006622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:18.024613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1130/2233] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:35:17 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:35:18.029654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:18.043998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135eb2c-5fef-4e64-ac13-9e54dbddea94 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:18.050797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:18.051714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:18.052149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:18.052611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] PATH_INFO: `/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14/users/8573cfc4ecda47ab8480769977aeac07/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:18.055745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ae6c7e6f846dba2b27407c6a30d14, user_id=8573cfc4ecda47ab8480769977aeac07, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:18.061193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f519fe2-778a-481d-913a-cdb936ce53ad admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:18.093645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1104/2234] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:35:18 2022] PUT /identity/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14/users/8573cfc4ecda47ab8480769977aeac07/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:19.477593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:19.479461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:19.495754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2181-39f1-4c78-b710-0bcefaaf26c2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:19.582386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:19.583061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:19.583350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:19.583653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] PATH_INFO: `/v3/users/c213e18f6fac4354bae2b6537d355eaa` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:19.586584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c213e18f6fac4354bae2b6537d355eaa)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:19.590181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:20.320115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c213e18f6fac4354bae2b6537d355eaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:20.321113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c213e18f6fac4354bae2b6537d355eaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:20.340210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c213e18f6fac4354bae2b6537d355eaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:20.347443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c213e18f6fac4354bae2b6537d355eaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:20.361554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c213e18f6fac4354bae2b6537d355eaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:20.523105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1131/2235] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:35:19 2022] DELETE /identity/v3/users/c213e18f6fac4354bae2b6537d355eaa => generated 0 bytes in 1043 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:20.524774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:20.525205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:20.525498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:20.546758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:20.548083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:20.625591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1105/2236] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:35:20.641919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:20.643357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:20.719653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57088a4f-2815-491e-a8f1-2bd08765082c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:20.724298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:20.725033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:20.725418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:20.725886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:20.737176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:20.738788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c5bec-2cbf-4988-b55f-8e6606f378b8 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:20.757257 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1132/2237] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:35:20.792289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:21.485191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:21.490562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:21.504445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5efc8-2bb4-4b66-8991-eb7d39cc7620 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:21.511555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:21.512208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:21.512595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:21.512830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] PATH_INFO: `/v3/projects/f3d6aac1edaf4aebad2c00577e26e4f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:21.513952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d6aac1edaf4aebad2c00577e26e4f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:21.516341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:21.719838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d6aac1edaf4aebad2c00577e26e4f8', 'request_id': 'req-76ef09d1-1000-40a3-ab94-ba97aac3630a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:21.726852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d6aac1edaf4aebad2c00577e26e4f8', 'request_id': 'req-76ef09d1-1000-40a3-ab94-ba97aac3630a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:21.733152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d6aac1edaf4aebad2c00577e26e4f8', 'request_id': 'req-76ef09d1-1000-40a3-ab94-ba97aac3630a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:21.903017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] The token cache is being invalidate because project f3d6aac1edaf4aebad2c00577e26e4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:35:21.904312 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:21.905044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:21.936993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1106/2238] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:35:21 2022] DELETE /identity/v3/projects/f3d6aac1edaf4aebad2c00577e26e4f8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:23.381488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:25.814039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c63cfb-68e7-46fd-9f88-937db0e1b170 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c63cfb-68e7-46fd-9f88-937db0e1b170 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c63cfb-68e7-46fd-9f88-937db0e1b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e6c63cfb-68e7-46fd-9f88-937db0e1b170 None None] MFA Rules not processed for user `8573cfc4ecda47ab8480769977aeac07`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6c63cfb-68e7-46fd-9f88-937db0e1b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:26.794857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1133/2239] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:26.798376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:26.798376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:26.798376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef09d1-1000-40a3-ab94-ba97aac3630a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:26.799569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:26.799569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:26.799569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:26.799569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:26.801559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:26.801559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:26.801559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476f7e7-62bd-41e3-be82-067cb3ebd044 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:26.801559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1107/2240] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:35:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:35:27.798418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:30.341369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:30.506861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:30.798861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:30.798861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:30.798861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:30.817064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] MFA Rules not processed for user `3b089f2cc22449329110fd3cfafecd53`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:30.818424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:30.826170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1134/2241] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:33.043262 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:34.364171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:35.169073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:36.501251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:37.672657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:38.883569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:39.775095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141e74-70b4-4568-8e84-7d3b3fdc5cd8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:39.775569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141e74-70b4-4568-8e84-7d3b3fdc5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:39.775789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141e74-70b4-4568-8e84-7d3b3fdc5cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:39.797121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b8141e74-70b4-4568-8e84-7d3b3fdc5cd8 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:39.798348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b8141e74-70b4-4568-8e84-7d3b3fdc5cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:39.857783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1108/2242] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:35:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:35:39.906481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:39.908328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:39.994003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bd4da-a618-4cdd-9184-716fe785ef27 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:39.999506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:40.000136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:40.000442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:40.000712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:40.011848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:40.013671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:40.023517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1135/2243] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:35:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:35:41.101601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:41.127411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821cb3-42b3-403b-9c44-1fc51144e287 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:41.127702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821cb3-42b3-403b-9c44-1fc51144e287 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:41.127977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821cb3-42b3-403b-9c44-1fc51144e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:41.148032 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d1821cb3-42b3-403b-9c44-1fc51144e287 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:41.150040 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d1821cb3-42b3-403b-9c44-1fc51144e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:41.158225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1109/2244] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:41.178728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:41.189831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64bef6-5cb8-4145-b0e2-e4e4fab02d76 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:41.195642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:41.196279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:41.196585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:41.196858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:41.205344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:41.206998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:41.218826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1136/2245] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:35:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:35:42.192986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:42.336227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deae930-b754-46c3-b87c-770d5dd05775 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:42.337199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deae930-b754-46c3-b87c-770d5dd05775 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:42.337984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deae930-b754-46c3-b87c-770d5dd05775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:42.357522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9deae930-b754-46c3-b87c-770d5dd05775 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:42.358718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9deae930-b754-46c3-b87c-770d5dd05775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:42.365995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1110/2246] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:42.379731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:42.396933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43176d82-1551-4219-b901-0a37b6d11df0 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:42.402669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:42.403280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:42.403564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:42.403825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:42.414283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:42.416469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:42.424760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1137/2247] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:35:43.656469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782380-1986-4861-8b35-5240139f042b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:43.656930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782380-1986-4861-8b35-5240139f042b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:43.657069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782380-1986-4861-8b35-5240139f042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:43.678406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2c782380-1986-4861-8b35-5240139f042b None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:43.679643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2c782380-1986-4861-8b35-5240139f042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:43.687930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1111/2248] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:35:43.700558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:43.710851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979ce1-3644-4893-8ec8-d4c6956a8a18 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:43.716309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:43.716978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:43.717282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:43.717590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:43.725620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:43.727193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf1f0d-0bd9-4a63-94f6-2fd333eb3e23 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:43.735646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1138/2249] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:35:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:35:44.632239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:45.204049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:46.820177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:47.734374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:48.195967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782380-1986-4861-8b35-5240139f042b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:48.197610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2c782380-1986-4861-8b35-5240139f042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:48.261185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782380-1986-4861-8b35-5240139f042b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:48.265873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:48.266511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:48.266846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:48.267122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] PATH_INFO: `/v3/users/75c919b5c8624a2ebad5bff722a3ff11` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:48.268298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c919b5c8624a2ebad5bff722a3ff11)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:48.270559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:48.549395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c919b5c8624a2ebad5bff722a3ff11'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:48.550055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c919b5c8624a2ebad5bff722a3ff11'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:48.654283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c919b5c8624a2ebad5bff722a3ff11'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:48.655024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c919b5c8624a2ebad5bff722a3ff11'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:48.662944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c919b5c8624a2ebad5bff722a3ff11'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:48.680122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1112/2250] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:35:48 2022] DELETE /identity/v3/users/75c919b5c8624a2ebad5bff722a3ff11 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:35:48.684948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:48.685308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:48.685617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:48.713440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:35:48.713440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:48.769210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1139/2251] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:35:48.778207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:48.779684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:48.837271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49386b-328d-4c06-bd90-d50b0dab7bd4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:48.843670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:48.843670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:48.843670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:48.843670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:48.851840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:48.853353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:48.862807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1113/2252] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:35:49.193124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:35:49.194985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:35:49.207533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41ba08-e084-4cd0-b306-94b3723421fa None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:35:49.212168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:35:49.212862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:35:49.213145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:35:49.213425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] PATH_INFO: `/v3/projects/fcbae11ef93f4951a3bc244ef9fd7a05` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:35:49.214626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbae11ef93f4951a3bc244ef9fd7a05)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:35:49.216583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:35:49.392615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbae11ef93f4951a3bc244ef9fd7a05', 'request_id': 'req-db5eb6b0-3fae-43f1-916f-0672a4681c90'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:49.399052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbae11ef93f4951a3bc244ef9fd7a05', 'request_id': 'req-db5eb6b0-3fae-43f1-916f-0672a4681c90'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:49.523343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:49.546666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbae11ef93f4951a3bc244ef9fd7a05', 'request_id': 'req-db5eb6b0-3fae-43f1-916f-0672a4681c90'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:49.558363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] The token cache is being invalidate because project fcbae11ef93f4951a3bc244ef9fd7a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:35:49.558648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:49.559265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-db5eb6b0-3fae-43f1-916f-0672a4681c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:35:49.560799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1140/2253] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:35:49 2022] DELETE /identity/v3/projects/fcbae11ef93f4951a3bc244ef9fd7a05 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:35:49.907232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:50.621629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:51.438507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:51.797300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:52.295320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:53.215655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:53.260975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:53.581159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:54.455668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:54.507589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:54.832499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:55.105629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:55.442858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:55.564038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:55.728739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:56.261333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:56.335665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:56.609589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:56.785950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:56.951155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:57.118437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:57.364732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:57.460561 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:58.043003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:58.180175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:58.534199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:58.564351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:58.752597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.049795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.128470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.558235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.791557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.905709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:35:59.993087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:00.341272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:00.465797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:00.821205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:01.815666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:01.998672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:02.409271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:02.724696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:36:02.726092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:36:02.799757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff93f07-bc82-45fe-a1d4-27b99ef9c591 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:36:02.856274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:02.859977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:36:02.860725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:36:02.861004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:36:02.868225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:36:02.868225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:36:02.874220 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:36:02.875856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:36:02.885792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1114/2254] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:36:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:36:03.215621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:03.347115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:03.528278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:03.769751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:13.883916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:18.337527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:36:18.338136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:36:18.338136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:36:18.357135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] MFA Rules not processed for user `4c6fffcc3c0d4c49848fa6c2f6b90130`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:36:18.358348 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:36:18.421049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1141/2255] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:36:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:36:24.117450 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:36:24.118916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:36:24.182168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dae9ad-a9aa-4d29-a888-e9117c67a294 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:36:24.185843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:36:24.186459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:36:24.186739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:36:24.187012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:36:24.189307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:36:24.251983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:36:24.253457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:36:24.263026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1115/2256] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:36:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:36:28.408793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:36:34.258662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=50 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:37:32.916293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:37:32.917714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:32.929584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7c537-b7de-496c-b9bc-bc00db78a0ac None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:37:32.933132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:37:32.933743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:32.934022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:32.934312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:32.936018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:33.011541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:37:33.012930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f52e3b-097d-4fd8-9a38-a7a9a54e3e4f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:37:33.025049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1142/2257] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:37:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:37:34.151729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:37:34.160331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154230a-5a99-47f1-a8e5-e858da88d3e7 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:37:34.163960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:37:34.164601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:34.164890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:34.165172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:34.166937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:34.180689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:37:34.182122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:37:34.193869 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1116/2258] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:37:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:37:39.276636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:39.276636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:39.276636 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:39.289957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:37:39.291167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:39.347916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1143/2259] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:37:39.506261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:37:39.507691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:39.576787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df581d-eb76-4efd-b130-6bae6fee212c service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:37:39.581840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:37:39.582448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:39.582725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:39.582993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:39.590904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:37:39.592590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:37:39.601314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1117/2260] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:37:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:37:45.539154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:37:45.541183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:45.558003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7bdc5-6ff3-4fbb-ac30-2d3d34786f77 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:37:45.563470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:37:45.564451 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:45.564750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:45.565712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:45.567523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:45.649277 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:37:45.651023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea5b6-b3da-486a-9206-8ab84a0b5a05 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:37:45.661445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1144/2261] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:37:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:37:49.597966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=38 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:37:55.655986 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:37:59.601455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:37:59.608241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47a47-754e-401b-b609-86b36d4a1451 service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:37:59.611896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:37:59.612584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:59.612805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:59.613077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:59.614811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:59.624911 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:37:59.626395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11aad7-8db6-425c-8054-16e22dc5addf service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:37:59.634403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1118/2262] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:37:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:37:59.968930 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25336bd-371f-4722-a4f6-e4b0f246bec9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:37:59.970682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25336bd-371f-4722-a4f6-e4b0f246bec9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:37:59.970682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25336bd-371f-4722-a4f6-e4b0f246bec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:37:59.989424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b25336bd-371f-4722-a4f6-e4b0f246bec9 None None] MFA Rules not processed for user `07e70b1c968443eab469777283b8fb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:37:59.990665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b25336bd-371f-4722-a4f6-e4b0f246bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:37:59.998900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1145/2263] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:37:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:00.001177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.001764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.005003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.005003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1119/2264] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:37:59 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Jul 26 16:38:00.019005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069875-b8ac-429d-9fdc-c4bbb4822598 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.019385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069875-b8ac-429d-9fdc-c4bbb4822598 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.019719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069875-b8ac-429d-9fdc-c4bbb4822598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.041844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bc069875-b8ac-429d-9fdc-c4bbb4822598 None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:00.043231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bc069875-b8ac-429d-9fdc-c4bbb4822598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.060457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1146/2265] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:00.078635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:00.081054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.154403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e517b6-9dde-4af7-a602-63d2723114d5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:00.159807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:00.160442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.160723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.160991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.171452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:00.172942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb004-9776-46ea-8fa8-3dabfb915010 service glance] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:00.183828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1120/2266] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:00.495697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31f6-950f-45a4-9d73-ee9d1d4617f8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.495987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31f6-950f-45a4-9d73-ee9d1d4617f8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.496272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b31f6-950f-45a4-9d73-ee9d1d4617f8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.497999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1147/2267] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:38:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:38:00.503815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f8a2-225c-4c9b-8fe6-543ba27f591d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.504144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f8a2-225c-4c9b-8fe6-543ba27f591d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.504525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f8a2-225c-4c9b-8fe6-543ba27f591d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.524411 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bab0f8a2-225c-4c9b-8fe6-543ba27f591d None None] MFA Rules not processed for user `bd7e03b6ec884bf2bf5039aefba4e5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:00.525619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bab0f8a2-225c-4c9b-8fe6-543ba27f591d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.532050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1121/2268] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:00.544086 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b31f6-950f-45a4-9d73-ee9d1d4617f8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:00.555817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b31f6-950f-45a4-9d73-ee9d1d4617f8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:00.560027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:00.560961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.561354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.561629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.570797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:00.572932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fe0ac-7afd-452f-85f6-516602fcf2e2 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:00.582325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1148/2269] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:00.679735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6346-5629-4552-be73-ee42da93945e None None] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.680294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6346-5629-4552-be73-ee42da93945e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.680294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6346-5629-4552-be73-ee42da93945e None None] PATH_INFO: `/` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.682576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1122/2270] 10.222.0.144 () {54 vars in 926 bytes} [Tue Jul 26 16:38:00 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Jul 26 16:38:00.691569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.698677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.698677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.711949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] MFA Rules not processed for user `6268372f3956497aae05aec64d41e848`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:00.713243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.723753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1149/2271] 10.222.0.144 () {58 vars in 1023 bytes} [Tue Jul 26 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:00.734786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c6346-5629-4552-be73-ee42da93945e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:00.747364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c6346-5629-4552-be73-ee42da93945e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:00.751235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:00.751908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.752188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.752501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.763792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:00.765287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2c20-6ec3-43cd-a6d8-a76281822cad service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:00.777867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1123/2272] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:00.841797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:00.843198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.858184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b0ddf-1279-4f78-8ff3-c572a3d97cd3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:00.863426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:00.864061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:00.864343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:00.864651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:00.866499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:00.879251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:00.880902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:00.892298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1150/2273] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:07.385166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:07.385660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:07.385762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:07.406834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:07.408136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:07.415308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1124/2274] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:38:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:07.427546 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:07.439039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab40383-56a3-4fdb-9ee6-3ea3d65c730b service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:07.444130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:07.444726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:07.445009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:07.445473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:07.464484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:07.466184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:07.475864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1151/2275] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 16:38:15.841214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:15.868148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:15.868148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017fe21-74d3-4281-862c-e638fbb5c415 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:15.870512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:15.871137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:15.871423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:15.871700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:15.873538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:15.948223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:15.949791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39390f4-f9a4-41a6-aa4f-ebcfbc69347c service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:15.960919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1125/2276] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:17.472580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=27 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:38:21.543510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:21.553322 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e2e4b-67b1-4907-b5a1-8e2af9361926 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:21.559440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:21.559440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:21.559440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:21.559440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:21.559901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:21.570194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:21.571609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:21.579692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1152/2277] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:25.956116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=72 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:38:31.576118 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:38:35.896361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:35.896864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:35.896973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:35.917019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:35.918215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:35.925347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1126/2278] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:38:35.937719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:35.949969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82a6e-6ac9-4e35-b531-9dfc90644922 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:35.955501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:35.956102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:35.956411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:35.956684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:35.964803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:35.966430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:35.974857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1153/2279] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:38:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:38:36.830992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:36.843838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056eb3c-b4f9-4e69-94f3-824716d098a9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:36.849797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:36.850421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:36.850695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:36.850967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:36.852792 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:36.864084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:36.865683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1781c-c7ba-444a-8690-2a56a3a3f208 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:36.874992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1127/2280] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:40.950993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:40.966747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6037c-93e6-4253-957e-7c54fcb66af6 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:40.972496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:40.973112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:40.973393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:40.973670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:40.976545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:40.989725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:40.991363 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:41.002691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1154/2281] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:38:46.870509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=9 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:38:50.997526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=12 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:38:54.755742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:54.756242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:54.756355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:54.775161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] MFA Rules not processed for user `4c6fffcc3c0d4c49848fa6c2f6b90130`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:38:54.776355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:54.783724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1128/2282] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:38:55.087175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:38:55.096871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1057de-08b4-46ab-a232-39420ed98ea1 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:38:55.100679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:38:55.101284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:38:55.101563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:38:55.101834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:38:55.103528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:38:55.113405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:38:55.114810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:38:55.122572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1155/2283] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:04.770193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:05.119062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:09.784764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:09.798310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde46235-fa93-46df-bea8-4e55602d7e57 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:09.804096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:09.804742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:09.805024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:09.805299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:09.807070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:09.820381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:09.821991 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:09.832941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1129/2284] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:15.454999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:15.468089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979e54-57a8-476c-a080-e383416c364c service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:15.474669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:15.475530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:15.475881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:15.476278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:15.478123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:15.500014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:15.500014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:15.507715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1156/2285] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:19.828402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=6 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:25.503216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:39.170266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:39.180767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c495c-7a5c-4ee3-bd9b-fc91226c850d service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:39.184618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:39.187115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:39.187115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:39.187115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:39.187626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:39.254029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:39.255459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:39.265570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1130/2286] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:43.418259 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:43.430394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0a3f-b7de-458d-9ca0-77f766f8cc79 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:43.435534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:43.436242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:43.436561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:43.436836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:43.439147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:43.451480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:43.453166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:43.463279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1157/2287] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:43.553513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:43.555044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:43.567387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cadc5-0cf5-4303-a8ba-03ec723ef31e service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:43.572045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:43.572712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:43.572997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:43.573274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] PATH_INFO: `/v3/users/07e70b1c968443eab469777283b8fb50` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:43.574544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e70b1c968443eab469777283b8fb50)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:43.576488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:43.727471 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e70b1c968443eab469777283b8fb50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:43.728133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e70b1c968443eab469777283b8fb50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:43.787900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e70b1c968443eab469777283b8fb50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:43.788560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e70b1c968443eab469777283b8fb50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:43.795580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-82278644-431d-47c7-9055-e6afac32a52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e70b1c968443eab469777283b8fb50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:43.812998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1131/2288] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:39:43 2022] DELETE /identity/v3/users/07e70b1c968443eab469777283b8fb50 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:39:44.054343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.067084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dd534-53a7-42f0-8646-bc9dd1175ce9 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.130962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.131599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.131877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.132147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] PATH_INFO: `/v3/projects/08a954085f8f454698c55faa9351ad6e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.133315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a954085f8f454698c55faa9351ad6e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.135198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.248710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a954085f8f454698c55faa9351ad6e', 'request_id': 'req-da48cfe7-15ae-4163-aca3-e3b9e0caf970'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:44.256054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a954085f8f454698c55faa9351ad6e', 'request_id': 'req-da48cfe7-15ae-4163-aca3-e3b9e0caf970'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:44.345549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a954085f8f454698c55faa9351ad6e', 'request_id': 'req-da48cfe7-15ae-4163-aca3-e3b9e0caf970'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:44.357718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] The token cache is being invalidate because project 08a954085f8f454698c55faa9351ad6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:39:44.358007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:44.358693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-da48cfe7-15ae-4163-aca3-e3b9e0caf970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:44.360249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1158/2289] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:39:44 2022] DELETE /identity/v3/projects/08a954085f8f454698c55faa9351ad6e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:39:44.365382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.365831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.366127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.387540 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:39:44.388863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:44.451117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1132/2290] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:39:44.459684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.460205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.460361 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.480946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:39:44.482145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:44.495423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1159/2291] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:39:44.499637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.509866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e4869-5d10-47be-9401-6546fad98e90 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.514565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.515223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.515505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.515787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.517263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:39:44.517580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.519171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.520752 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:44.533213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1133/2292] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:39:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:39:44.537376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.547045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9678-411f-4790-b000-72f6a02f6b98 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.551707 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.552342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.552654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.552923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.553922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.556734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.619699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1160/2293] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:39:44 2022] POST /identity/v3/projects => generated 368 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:39:44.623992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.635253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12fdf6-7f25-49a1-9814-509277a77d56 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.690820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.691437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.691720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.691998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.692697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.694607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.782878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1134/2294] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:39:44 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:39:44.787222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.798002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff8cd-b344-4ff1-b6be-4f6163456e85 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.802716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.803315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.803667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.804055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.804781 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.806503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.816952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1161/2295] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:39:44 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:39:44.820864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.830828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7256f3-9316-4345-b6ff-a061311f31d1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.835406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.836004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.836289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.836592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.837283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.838867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e2f75-692c-4944-80a6-4b466e8c5135 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:44.850318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1135/2296] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:39:44 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:39:44.854289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:44.864720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faf9bf-8158-49c2-acf5-1285b7117e85 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:44.869274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:44.869966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:44.870193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:44.870463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] PATH_INFO: `/v3/projects/7be780bbb9e0444f96cdc0c4dbf8e84c/users/7db983485b1f470daf212f8b8cb97df4/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:44.885025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be780bbb9e0444f96cdc0c4dbf8e84c, user_id=7db983485b1f470daf212f8b8cb97df4, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:44.887634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:45.003429 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1162/2297] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:39:44 2022] PUT /identity/v3/projects/7be780bbb9e0444f96cdc0c4dbf8e84c/users/7db983485b1f470daf212f8b8cb97df4/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:39:45.014571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:45.014857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:45.015196 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:45.043488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] MFA Rules not processed for user `7db983485b1f470daf212f8b8cb97df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:39:45.044878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:45.142746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1136/2298] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:39:45.153726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:45.155414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:45.169918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:45.211157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8340a-2932-40a8-9dba-7e6bd6c62e08 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:45.214817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:45.215417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:45.215697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:45.215972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:45.224633 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:45.226050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:45.236597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1163/2299] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:54.441559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:54.443000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:54.504849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:54.520806 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9983b0d-5a4b-4891-a521-f634192d87e5 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:54.526288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:54.527165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:54.527447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:54.527723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:54.529495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:54.590999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:54.592688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbce96-3ebe-4d14-a61a-f9e764aa018d service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:54.602391 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1137/2300] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:55.264474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=20 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:39:58.649559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:58.710444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d517-91bf-457f-afb4-7ffc040079b0 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:58.714948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:58.715570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:58.715849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:58.716119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] PATH_INFO: `/v3/users/7db983485b1f470daf212f8b8cb97df4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:58.717295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db983485b1f470daf212f8b8cb97df4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:58.719198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:58.931710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db983485b1f470daf212f8b8cb97df4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:58.932364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db983485b1f470daf212f8b8cb97df4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:58.939236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db983485b1f470daf212f8b8cb97df4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:58.939871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db983485b1f470daf212f8b8cb97df4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:58.951549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db983485b1f470daf212f8b8cb97df4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.059344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1164/2301] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:39:58 2022] DELETE /identity/v3/users/7db983485b1f470daf212f8b8cb97df4 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:39:59.064093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:59.064623 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:59.064823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:59.084935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:39:59.086146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:59.143893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1138/2302] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:39:59.153917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:59.155334 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:59.212817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440e3d3-72f8-4eeb-8364-3626c0982053 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:59.216531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:59.217148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:59.217430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:59.217706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:59.227484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:59.228948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2237f-dc33-44e7-9a22-08a97bae825e service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:59.238370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1165/2303] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:39:59.745440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:39:59.746910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:39:59.759600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac9e00-3870-493b-bd24-66ace0932fb0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:39:59.776994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:39:59.776994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:59.776994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:59.776994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] PATH_INFO: `/v3/projects/7be780bbb9e0444f96cdc0c4dbf8e84c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:39:59.777559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be780bbb9e0444f96cdc0c4dbf8e84c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:39:59.777559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:39:59.888879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be780bbb9e0444f96cdc0c4dbf8e84c', 'request_id': 'req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.895093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be780bbb9e0444f96cdc0c4dbf8e84c', 'request_id': 'req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.900957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be780bbb9e0444f96cdc0c4dbf8e84c', 'request_id': 'req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.978736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] The token cache is being invalidate because project 7be780bbb9e0444f96cdc0c4dbf8e84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:39:59.979037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.979758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dd1e1fb3-0ff8-4f54-889d-ce29d0a82e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:39:59.981378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1139/2304] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:39:59 2022] DELETE /identity/v3/projects/7be780bbb9e0444f96cdc0c4dbf8e84c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:39:59.986734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:39:59.987294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:39:59.987294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.008815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:00.009972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:00.029597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:00.073511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1166/2305] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:00.082108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.082681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.082681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.112156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:00.113412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:00.122308 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1140/2306] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:40:00.130312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.136430 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b07d1-ae43-4a96-82bc-aee09b325f08 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.141275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.142459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.142735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.143004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.143703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:40:00.144004 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.145664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.159203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1167/2307] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:40:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:40:00.163403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.173091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab0f29-e4d1-46ea-94c5-c14aeb40b109 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.177745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.178353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.178630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.178899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.179559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.181467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.196778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:00.322148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1141/2308] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:00 2022] POST /identity/v3/projects => generated 366 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:00.326901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.337502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a32151-baf5-4309-94d3-5e17755e095f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.406668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.407310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.407592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.407868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.408559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.410496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.522150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1168/2309] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:00 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:00.526357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.536907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5204-e1f8-47cd-909d-7ca96360fe21 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.543249 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.543851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.544134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.544442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.545148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.546768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.557963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1142/2310] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:00 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:00.562025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.580106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d666f84-623f-4101-ac5b-ed16fbcfcff7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.585013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.585830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.585958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.586179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.586872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.588551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f5e13-b817-447b-99aa-0c765541024f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.608757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1169/2311] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:00 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:00.613856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.624229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7d4a-5fe9-4a7f-868f-8e79e5e94c96 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.629617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.630408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.630694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.630996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] PATH_INFO: `/v3/projects/7c547b084fb744b5b808d3f544c30e9a/users/1c385cbfdfbf463db36319d3d79d5511/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.646452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c547b084fb744b5b808d3f544c30e9a, user_id=1c385cbfdfbf463db36319d3d79d5511, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.649147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:00.756482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1143/2312] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:00 2022] PUT /identity/v3/projects/7c547b084fb744b5b808d3f544c30e9a/users/1c385cbfdfbf463db36319d3d79d5511/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:00.768410 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29761-987f-4523-9641-229a43a091b4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.768699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29761-987f-4523-9641-229a43a091b4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.768979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df29761-987f-4523-9641-229a43a091b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.801923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5df29761-987f-4523-9641-229a43a091b4 None None] MFA Rules not processed for user `1c385cbfdfbf463db36319d3d79d5511`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:00.801923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5df29761-987f-4523-9641-229a43a091b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:00.871320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1170/2313] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:00.875952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:00.946152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186465c-76a5-4e80-91a2-809449f983cf admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:00.951397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:00.952029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:00.952380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:00.956541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:00.956541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:00.956541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.057979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1144/2314] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:00 2022] POST /identity/v3/projects => generated 364 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:01.062094 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df29761-987f-4523-9641-229a43a091b4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.076300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df29761-987f-4523-9641-229a43a091b4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.095873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:01.170045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.171218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.171586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.171993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.173326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.176087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.298193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1171/2315] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:01 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:01.299331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.311704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b376f9-9ca7-4ca0-a016-8edd9263c963 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.318655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.319283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.319692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.331689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.331689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.331689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.343882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1145/2316] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:01 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:01.348242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.358391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e4142-31bc-4be3-824e-78a8ef1850d8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.363689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.364452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.364800 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.365140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] PATH_INFO: `/v3/projects/c43a508b47054499b22bee3092e7ce85/users/5a1619f8fc1c4e48a3702b90fb50db42/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.386626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c43a508b47054499b22bee3092e7ce85, user_id=5a1619f8fc1c4e48a3702b90fb50db42, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.389849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.543275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1172/2317] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:01 2022] PUT /identity/v3/projects/c43a508b47054499b22bee3092e7ce85/users/5a1619f8fc1c4e48a3702b90fb50db42/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:01.547499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.558436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906d713-78a4-4f3e-bb9c-8fda20aca3f7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.639872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.640509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.640796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.641069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.641768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.643382 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728bdd2-a9ef-4080-94ba-6e4d6f7ed202 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.658025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1146/2318] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:01 2022] GET /identity/v3/roles => generated 1427 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:01.659318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.671204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cec4a8-a1c5-4202-b43f-23dfcc3f33b5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.677292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.677966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.678819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.678819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] PATH_INFO: `/v3/projects/c43a508b47054499b22bee3092e7ce85/users/5a1619f8fc1c4e48a3702b90fb50db42/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.681127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c43a508b47054499b22bee3092e7ce85, user_id=5a1619f8fc1c4e48a3702b90fb50db42, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.688212 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.696372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1173/2319] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:01 2022] PUT /identity/v3/projects/c43a508b47054499b22bee3092e7ce85/users/5a1619f8fc1c4e48a3702b90fb50db42/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:01.707104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.707476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.707822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.739740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] MFA Rules not processed for user `5a1619f8fc1c4e48a3702b90fb50db42`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:01.741005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:01.761074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:01.814765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1147/2320] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:01.826408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:01.828224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:01.888960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95cbba-9a06-4ef7-880f-561a987e502c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:01.893289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:01.893848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:01.894130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:01.894404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:01.903832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:01.905295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68a1b6-6af0-4761-9334-9cf2517d7444 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:01.916627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1174/2321] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:02.179374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:02.189875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874eb4-95c7-49e6-ad2a-99f6ec769e42 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:02.193443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:02.194046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:02.194356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:02.194633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:02.251188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:02.252640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0568d-017a-482a-a36b-0512ab78f835 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:02.261467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1148/2322] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:07.769769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:07.770320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:07.770320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:07.789847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] MFA Rules not processed for user `09de5ea60b9943adacc765b5ecdccb71`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:07.791050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:07.848497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1175/2323] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:09.884029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48372ee5-7817-4d41-95a0-c43216b30553 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:09.884598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48372ee5-7817-4d41-95a0-c43216b30553 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:09.884598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48372ee5-7817-4d41-95a0-c43216b30553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:09.903645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-48372ee5-7817-4d41-95a0-c43216b30553 None None] MFA Rules not processed for user `1b9d76409f834b478981c0bbc0c3d4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:09.904882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-48372ee5-7817-4d41-95a0-c43216b30553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:09.959183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1149/2324] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:40:17.030053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:17.050535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:17.094911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1a91-66d8-47dc-92d7-aea0e812fcce None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:17.099533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:17.100169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:17.100483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:17.100760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] PATH_INFO: `/v3/users/1c385cbfdfbf463db36319d3d79d5511` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:17.101919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c385cbfdfbf463db36319d3d79d5511)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:17.103835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:17.321668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c385cbfdfbf463db36319d3d79d5511'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:17.322288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c385cbfdfbf463db36319d3d79d5511'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:17.329218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c385cbfdfbf463db36319d3d79d5511'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:17.329850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c385cbfdfbf463db36319d3d79d5511'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:17.341353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c385cbfdfbf463db36319d3d79d5511'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:17.411269 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1176/2325] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:17 2022] DELETE /identity/v3/users/1c385cbfdfbf463db36319d3d79d5511 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:17.415805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:17.416285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:17.416609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:17.436617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:17.437820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:17.487668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:17.497077 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1150/2326] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:17.506680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:17.513822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:17.565378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116bc8-8744-40b7-ad99-560f89164d62 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:17.569100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:17.569720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:17.570104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:17.570528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:17.581171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:17.582536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:17.592041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1177/2327] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:18.037086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:18.038566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:18.051896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afbc90-947c-4c86-a9ad-cc6935a45d3d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:18.056973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:18.058397 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:18.058926 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:18.059328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] PATH_INFO: `/v3/projects/7c547b084fb744b5b808d3f544c30e9a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:18.060773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c547b084fb744b5b808d3f544c30e9a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:18.063831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:18.194771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c547b084fb744b5b808d3f544c30e9a', 'request_id': 'req-dfaa443b-bb26-4c37-8325-e964ef3bb722'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.200958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c547b084fb744b5b808d3f544c30e9a', 'request_id': 'req-dfaa443b-bb26-4c37-8325-e964ef3bb722'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.207002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c547b084fb744b5b808d3f544c30e9a', 'request_id': 'req-dfaa443b-bb26-4c37-8325-e964ef3bb722'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.292095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] The token cache is being invalidate because project 7c547b084fb744b5b808d3f544c30e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:18.292412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.293026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.294559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1151/2328] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:18 2022] DELETE /identity/v3/projects/7c547b084fb744b5b808d3f544c30e9a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:18.298608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:18.308129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05569a6-4e36-4624-82c0-360962c5e5c7 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:18.347843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:18.365557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:18.366185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:18.366464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:18.366764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] PATH_INFO: `/v3/users/5a1619f8fc1c4e48a3702b90fb50db42` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:18.367913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1619f8fc1c4e48a3702b90fb50db42)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:18.369863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:18.586538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1619f8fc1c4e48a3702b90fb50db42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.587190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1619f8fc1c4e48a3702b90fb50db42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.593949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1619f8fc1c4e48a3702b90fb50db42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.594579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1619f8fc1c4e48a3702b90fb50db42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.606116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b300088e-89f5-4807-aace-bee8f51b382a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1619f8fc1c4e48a3702b90fb50db42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:18.702943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1178/2329] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:18 2022] DELETE /identity/v3/users/5a1619f8fc1c4e48a3702b90fb50db42 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:19.329209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:19.332625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:19.344332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa443b-bb26-4c37-8325-e964ef3bb722 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:19.401689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:19.402271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.402807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.403084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] PATH_INFO: `/v3/projects/c43a508b47054499b22bee3092e7ce85` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.404247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c43a508b47054499b22bee3092e7ce85)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:19.406223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:19.510235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c43a508b47054499b22bee3092e7ce85', 'request_id': 'req-ec082205-9883-4be4-baf1-bf8cf91cd34a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:19.516377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c43a508b47054499b22bee3092e7ce85', 'request_id': 'req-ec082205-9883-4be4-baf1-bf8cf91cd34a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:19.522142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c43a508b47054499b22bee3092e7ce85', 'request_id': 'req-ec082205-9883-4be4-baf1-bf8cf91cd34a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:19.612058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] The token cache is being invalidate because project c43a508b47054499b22bee3092e7ce85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:19.612368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:19.613994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ec082205-9883-4be4-baf1-bf8cf91cd34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:19.615689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1152/2330] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:19 2022] DELETE /identity/v3/projects/c43a508b47054499b22bee3092e7ce85 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:19.621863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.622386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.622581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.642910 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:19.649296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:19.706048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1179/2331] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:40:19.713884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.714205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.714487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.736643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:19.739955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:19.749641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1153/2332] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:19.753980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:19.764659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94081850-0b63-45bc-9083-8a045b52eeea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:19.769852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:19.770465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.770863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.771147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.771844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:40:19.772154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:19.773796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:19.786054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1180/2333] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:40:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:40:19.790356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:19.795959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:19.800405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171683-a6c4-4a4b-8fe4-7830a33db8a8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:19.805052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:19.805666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.805946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.806221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.806931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:19.809372 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:19.914060 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1154/2334] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:19 2022] POST /identity/v3/projects => generated 400 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:19.915160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:19.925435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2eb39-b0e6-4b64-a1cc-8ed8ca5931fd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:19.969100 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:19.982878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:19.984517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:19.984517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:19.984517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:19.984862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:19.986828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.085053 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1181/2335] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:19 2022] POST /identity/v3/users => generated 301 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:20.087289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.098296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81386-5033-454a-b79a-f51ebdbfc63a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.103076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.103693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.103972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.104245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.104998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.106575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.116619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1155/2336] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:20 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:20.121145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.132044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460bfc-9410-4a89-83fe-ff3ae03f6bde admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.136773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.137407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.137692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.137968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.138659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.140254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20aa7e0-0c4e-4426-b123-ee2ab9cb7d34 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.151754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1182/2337] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:20 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:20.156200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.167012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8152-d41a-437b-95cf-f4a835a678e7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.172307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.173182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.173377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.173579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] PATH_INFO: `/v3/projects/a9e7d3777d7d4f868c1377bc955679f3/users/0702d6a8b8a94415b295191cf61cac39/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.193624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e7d3777d7d4f868c1377bc955679f3, user_id=0702d6a8b8a94415b295191cf61cac39, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.196422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.300454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1156/2338] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:20 2022] PUT /identity/v3/projects/a9e7d3777d7d4f868c1377bc955679f3/users/0702d6a8b8a94415b295191cf61cac39/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:20.312169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.312623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.312969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.359487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] MFA Rules not processed for user `0702d6a8b8a94415b295191cf61cac39`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:20.361226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:20.428138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1183/2339] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:20.432514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.494107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f04a9-b659-4cf1-bfe3-e3297b95565c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.498850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.499448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.499730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.500008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.501218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.503223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.603507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1157/2340] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:20 2022] POST /identity/v3/projects => generated 402 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:40:20.607604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.618966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95022-f21a-427a-9e23-9cf1bb9bc82a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.677940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.678691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.678830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.679102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.679775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.681742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.806089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1184/2341] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:20 2022] POST /identity/v3/users => generated 308 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:20.810234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.821068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afedfa4-1bff-46f0-a8eb-68de1848aacf admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.825547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.826149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.826428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.826701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.827497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.829664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.840015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1158/2342] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:20 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:40:20.843990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.854407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d0e3-2d5f-4ae2-9b9e-db9f668dab32 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:20.859025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:20.859691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:20.859973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:20.860249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] PATH_INFO: `/v3/projects/33e65814488f45309d1161af0429d02f/users/e639025b48c345858f0985c5d7d8578f/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:20.874697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e65814488f45309d1161af0429d02f, user_id=e639025b48c345858f0985c5d7d8578f, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:20.877423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:20.950924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1185/2343] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:20 2022] PUT /identity/v3/projects/33e65814488f45309d1161af0429d02f/users/e639025b48c345858f0985c5d7d8578f/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:20.955455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:20.965596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f66c7-16ce-4359-8ef0-4384c2a5dcad admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:21.022259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:21.022994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:21.023289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:21.023660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:21.024355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:21.026012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d41ed-21d3-4f5c-98a3-147b00d2dfe7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:21.036304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1159/2344] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:20 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:21.040349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:21.050663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732c9f-14d8-4223-9198-d56ce0f4a8a9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:21.055236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:21.055884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:21.056164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:21.056464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] PATH_INFO: `/v3/projects/33e65814488f45309d1161af0429d02f/users/e639025b48c345858f0985c5d7d8578f/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:21.058365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e65814488f45309d1161af0429d02f, user_id=e639025b48c345858f0985c5d7d8578f, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:21.061010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:21.071520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1186/2345] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:21 2022] PUT /identity/v3/projects/33e65814488f45309d1161af0429d02f/users/e639025b48c345858f0985c5d7d8578f/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:21.082168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:21.082456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:21.082734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:21.109705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] MFA Rules not processed for user `e639025b48c345858f0985c5d7d8578f`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:21.110976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:21.115801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:21.177111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1160/2346] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:21.189464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:21.191149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:21.201220 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:21.248622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089fff5-ac68-4e98-92d0-1e2ca901c5dc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:21.252896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:21.255696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:21.255696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:21.255696 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:21.263151 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:21.264613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:21.274431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1187/2347] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:21.490744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:21.500334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d462-b985-452a-944b-b08e46d059bb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:21.504630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:21.509718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:21.509718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:21.509718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:21.571370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:21.572848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2291fc9-494b-40bf-801d-77ad84bd15dc service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:21.584489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1161/2348] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:28.188008 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:28.260260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e1c71-f366-4fc6-8600-af867aed3c71 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:28.265136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:28.265891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:28.266745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:28.267116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] PATH_INFO: `/v3/users/0702d6a8b8a94415b295191cf61cac39` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:28.268447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0702d6a8b8a94415b295191cf61cac39)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:28.270574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:28.504838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0702d6a8b8a94415b295191cf61cac39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:28.505514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0702d6a8b8a94415b295191cf61cac39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:28.513114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0702d6a8b8a94415b295191cf61cac39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:28.513776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0702d6a8b8a94415b295191cf61cac39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:28.529304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0702d6a8b8a94415b295191cf61cac39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:28.631099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1188/2349] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:28 2022] DELETE /identity/v3/users/0702d6a8b8a94415b295191cf61cac39 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:28.640972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:28.641967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:28.642245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:28.664477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:28.665704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:28.734194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1162/2350] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:40:28.744333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:28.745882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:28.777421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:28.805332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944927d-83b8-4546-bd2e-812013d79f6c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:28.809274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:28.809835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:28.810111 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:28.810385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:28.821309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:28.822817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:28.833935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1189/2351] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:29.336057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:29.337669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:29.389647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:29.422005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02591-a456-45e0-bb14-8d76fc685e63 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:29.426656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:29.427296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:29.427582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:29.427855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] PATH_INFO: `/v3/projects/a9e7d3777d7d4f868c1377bc955679f3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:29.429026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e7d3777d7d4f868c1377bc955679f3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:29.430925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:29.623597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e7d3777d7d4f868c1377bc955679f3', 'request_id': 'req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:29.630373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e7d3777d7d4f868c1377bc955679f3', 'request_id': 'req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:29.635842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e7d3777d7d4f868c1377bc955679f3', 'request_id': 'req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:29.733316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] The token cache is being invalidate because project a9e7d3777d7d4f868c1377bc955679f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:29.733833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:29.739044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:29.739044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1163/2352] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:29 2022] DELETE /identity/v3/projects/a9e7d3777d7d4f868c1377bc955679f3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:29.740524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:29.785889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70e753-007f-4b01-8ae8-d9f480cc7c80 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:29.842681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:29.843329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:29.843640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:29.843917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] PATH_INFO: `/v3/users/e639025b48c345858f0985c5d7d8578f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:29.845147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e639025b48c345858f0985c5d7d8578f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:29.847138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:30.021740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e639025b48c345858f0985c5d7d8578f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.022398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e639025b48c345858f0985c5d7d8578f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.029661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e639025b48c345858f0985c5d7d8578f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.030338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e639025b48c345858f0985c5d7d8578f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.042656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-66255432-fec9-4f44-a12d-2a506a58b215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e639025b48c345858f0985c5d7d8578f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.145911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1190/2353] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:29 2022] DELETE /identity/v3/users/e639025b48c345858f0985c5d7d8578f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:30.678524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:30.680307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:30.694151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4413a-b1d2-4e47-bf70-33b5f6db0dd2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:30.759269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:30.770914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:30.770914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:30.770914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] PATH_INFO: `/v3/projects/33e65814488f45309d1161af0429d02f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:30.770914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e65814488f45309d1161af0429d02f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:30.770914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:30.889069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e65814488f45309d1161af0429d02f', 'request_id': 'req-332edceb-2ac0-47bd-ab9a-d1b17daaa750'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.895264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e65814488f45309d1161af0429d02f', 'request_id': 'req-332edceb-2ac0-47bd-ab9a-d1b17daaa750'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.901416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e65814488f45309d1161af0429d02f', 'request_id': 'req-332edceb-2ac0-47bd-ab9a-d1b17daaa750'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:30.904575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:31.009737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] The token cache is being invalidate because project 33e65814488f45309d1161af0429d02f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:31.010095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:31.010805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-332edceb-2ac0-47bd-ab9a-d1b17daaa750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:31.012724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1164/2354] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:30 2022] DELETE /identity/v3/projects/33e65814488f45309d1161af0429d02f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:31.020677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.020677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.020677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.046652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:31.093767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:31.103491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:31.162441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1191/2355] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:31.171120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.171404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.171677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.191943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:31.193504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:31.202524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1165/2356] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:31.207289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.217644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedda251-fea4-43dd-946e-7ded7f9a9cf7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.222878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.226107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.226107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.226107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.226107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:40:31.226107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.227506 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.242075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1192/2357] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:40:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:40:31.246424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.256657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e2075-a6fe-462a-b53c-f777055bc2c0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.263432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.264128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.264509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.264781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.265636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.268168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.386321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1166/2358] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:31 2022] POST /identity/v3/projects => generated 388 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:31.390627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.401818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00006cb3-2b17-498e-ab00-0aeab950d5e6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.464835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.465297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.465579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.465858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.467442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.468557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.623824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1193/2359] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:31 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:31.627959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.639664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4784f-5074-4c24-976b-a1812a12acfe admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.644601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.645215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.645493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.645762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.646455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.648066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.664130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1167/2360] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:31 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:40:31.665077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.676895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65112a-4471-471f-8060-3bb1ce736d18 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.692685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.692685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.692685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.692685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.693110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.693110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd35b33-4fa7-4f1a-a54a-b30ad037e260 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.697461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1194/2361] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:31 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:31.701722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:31.711607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bddfab-7829-45f5-9b2e-de5e47dbbba1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:31.716204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:31.716892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.717172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.717445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] PATH_INFO: `/v3/projects/97bf89fd348a42988eecf5db51b3a9ef/users/69442d0ad9804cdfbbccc0d273f66fec/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.734095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bf89fd348a42988eecf5db51b3a9ef, user_id=69442d0ad9804cdfbbccc0d273f66fec, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:31.737386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:31.877010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1168/2362] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:31 2022] PUT /identity/v3/projects/97bf89fd348a42988eecf5db51b3a9ef/users/69442d0ad9804cdfbbccc0d273f66fec/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:31.893216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:31.893216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:31.893216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:31.930015 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] MFA Rules not processed for user `69442d0ad9804cdfbbccc0d273f66fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:31.939194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:31.981580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:31.999382 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1195/2363] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:32.003966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.071287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ea2ac-76a1-42f4-a88e-e34fe94b8582 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.077363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.078029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.078338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.078621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.079343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.081354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.199254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1169/2364] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:32 2022] POST /identity/v3/projects => generated 388 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:32.203470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.214619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a318af-0470-4be2-a05c-54b97d9343f5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.273127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.273656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.273935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.274213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.274874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.276825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.367735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1196/2365] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:32 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:32.371694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.381968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f395c2-5b65-4d8e-b2e9-4629a4c8e91f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.386719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.387328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.387637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.387916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.388648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.390238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.400804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1170/2366] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:32 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:32.405014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.415529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55248bd-4368-4710-9450-10bef7b2a85a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.420112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.420795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.421077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.421357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] PATH_INFO: `/v3/projects/f00da22f40de4610b82583811e03e760/users/b0e2e3c983a24be0b165c48d6bda39cc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.436722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00da22f40de4610b82583811e03e760, user_id=b0e2e3c983a24be0b165c48d6bda39cc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.439394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.539764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1197/2367] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:32 2022] PUT /identity/v3/projects/f00da22f40de4610b82583811e03e760/users/b0e2e3c983a24be0b165c48d6bda39cc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:32.543678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.554298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc4a7-09d6-4bda-a177-6e12d161644d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.611371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.612029 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.612367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.612675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.613368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.614954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381c821-7d58-4e51-a066-9ec99ea3beec admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.620041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:32.625089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1171/2368] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:32 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:32.629301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.639754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ab2a9-b06c-4dd5-abce-99d7d24931ef admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.644908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.645560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.645843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.646155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] PATH_INFO: `/v3/projects/f00da22f40de4610b82583811e03e760/users/b0e2e3c983a24be0b165c48d6bda39cc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.648105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00da22f40de4610b82583811e03e760, user_id=b0e2e3c983a24be0b165c48d6bda39cc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.650764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.661536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1198/2369] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:32 2022] PUT /identity/v3/projects/f00da22f40de4610b82583811e03e760/users/b0e2e3c983a24be0b165c48d6bda39cc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:40:32.671849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.672139 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.672444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.702854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] MFA Rules not processed for user `b0e2e3c983a24be0b165c48d6bda39cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:32.704625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:32.773317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1172/2370] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:32.787227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:32.788938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:32.859446 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33287d-ca5b-4598-98f1-c36149b074b9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:32.862990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:32.863583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:32.863859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:32.864136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:32.872845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:32.874258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3db7e-8059-4e4e-be21-611c4166d0f2 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:32.883409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1199/2371] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:34.238083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:34.301569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4918b6-1b72-4107-b2a5-bdcc3388fceb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:34.306349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:34.306994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:34.307299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:34.307554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] PATH_INFO: `/v3/users/69442d0ad9804cdfbbccc0d273f66fec` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:34.308810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] RBAC: Authorizing `identity:delete_user(user_id=69442d0ad9804cdfbbccc0d273f66fec)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:34.310695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:34.848068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69442d0ad9804cdfbbccc0d273f66fec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:34.848741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69442d0ad9804cdfbbccc0d273f66fec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:34.972797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69442d0ad9804cdfbbccc0d273f66fec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:34.973431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69442d0ad9804cdfbbccc0d273f66fec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:34.980834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69442d0ad9804cdfbbccc0d273f66fec'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:34.998786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1173/2372] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:34 2022] DELETE /identity/v3/users/69442d0ad9804cdfbbccc0d273f66fec => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:35.003933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:35.004411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:35.004527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:35.023313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:35.024594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:35.041150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:35.086073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1200/2373] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:35.096690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:35.098241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:35.104706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:35.155427 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15b3f-59f8-4811-a3dd-e5270cf6025a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:35.159026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:35.159642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:35.159928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:35.160205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:35.169173 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:35.170599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:35.180523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1174/2374] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:35.718816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:35.719153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:35.731441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb41ab8-0890-4076-8bb4-25fa1fb433f5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:35.736856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:35.737499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:35.737777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:35.738049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] PATH_INFO: `/v3/projects/97bf89fd348a42988eecf5db51b3a9ef` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:35.739206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bf89fd348a42988eecf5db51b3a9ef)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:35.741144 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:35.870480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bf89fd348a42988eecf5db51b3a9ef', 'request_id': 'req-912369d5-0133-450f-8de3-8b6027c3ff9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:35.877022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bf89fd348a42988eecf5db51b3a9ef', 'request_id': 'req-912369d5-0133-450f-8de3-8b6027c3ff9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.105267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bf89fd348a42988eecf5db51b3a9ef', 'request_id': 'req-912369d5-0133-450f-8de3-8b6027c3ff9a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.122186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] The token cache is being invalidate because project 97bf89fd348a42988eecf5db51b3a9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:36.122484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.123145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.124703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1201/2375] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:35 2022] DELETE /identity/v3/projects/97bf89fd348a42988eecf5db51b3a9ef => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:36.128709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:36.138569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d4965-8b5a-46f5-abd1-9536514efd17 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:36.195124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:36.195758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:36.196037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:36.196306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] PATH_INFO: `/v3/users/b0e2e3c983a24be0b165c48d6bda39cc` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:36.197613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e2e3c983a24be0b165c48d6bda39cc)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:36.199510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:36.491811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e2e3c983a24be0b165c48d6bda39cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.492504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e2e3c983a24be0b165c48d6bda39cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.605700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e2e3c983a24be0b165c48d6bda39cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.606341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e2e3c983a24be0b165c48d6bda39cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.614185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-44420118-77cf-4960-8cf2-0d2a4980af6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e2e3c983a24be0b165c48d6bda39cc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:36.631881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1175/2376] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:36 2022] DELETE /identity/v3/users/b0e2e3c983a24be0b165c48d6bda39cc => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:37.201729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:37.203195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:37.216723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912369d5-0133-450f-8de3-8b6027c3ff9a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:37.274685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:37.275318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:37.275597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:37.275870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] PATH_INFO: `/v3/projects/f00da22f40de4610b82583811e03e760` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:37.277038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00da22f40de4610b82583811e03e760)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:37.279399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:37.461727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:37.462835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00da22f40de4610b82583811e03e760', 'request_id': 'req-c78d0f85-75b6-4834-9122-8c54c466773e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:37.468828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00da22f40de4610b82583811e03e760', 'request_id': 'req-c78d0f85-75b6-4834-9122-8c54c466773e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:37.687383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00da22f40de4610b82583811e03e760', 'request_id': 'req-c78d0f85-75b6-4834-9122-8c54c466773e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:37.698059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] The token cache is being invalidate because project f00da22f40de4610b82583811e03e760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:37.698344 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:37.702976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c78d0f85-75b6-4834-9122-8c54c466773e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:37.702976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1202/2377] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:37 2022] DELETE /identity/v3/projects/f00da22f40de4610b82583811e03e760 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:37.705677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:37.706129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:37.706222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:37.751643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:37.751643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:37.751643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:37.786406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1176/2378] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:37.851791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:37.852079 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:37.852357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:37.871677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:37.872917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:37.883408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1203/2379] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:37.888098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:37.899212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8decea-9f5d-4d45-b888-b8e87e74c99c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:37.909269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:37.909269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:37.909269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:37.909269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:37.909627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:40:37.909627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:37.909627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:37.920003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1177/2380] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:40:37.924371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:37.933625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a6a9-8444-4639-a7ca-32c68798c2c0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:37.938229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:37.938831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:37.939110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:37.939383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:37.940036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:37.941998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.088331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1204/2381] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:37 2022] POST /identity/v3/projects => generated 376 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:38.092532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:38.103537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43eaf-ea9b-4668-9da4-81bccd74880a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:38.163337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:38.164075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.164259 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.164521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.165200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:38.167243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.343503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1178/2382] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:38 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:38.347549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:38.361927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaace8cb-63b3-44d4-8551-ecfa3379aa74 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:38.366513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:38.367127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.367407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.367683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.368376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:38.369998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.383645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1205/2383] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:38 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:38.384718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:38.396264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245852bb-d118-4124-8906-aac7b31fe7ee admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:38.401001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:38.401603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.401885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.402160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.402858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:38.404781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da2829-e5a2-485c-92d3-8c1e5f5acb36 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.419454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1179/2384] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:38 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:40:38.423737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:38.434403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16e33-0f69-4af9-878e-6bd8763cff23 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:38.443409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:38.444076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.444356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.444665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] PATH_INFO: `/v3/projects/b40cfbfeef1b4a98982e7dc3ed3ac6d4/users/72860d165fe246009c17df0e53800f50/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.461107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40cfbfeef1b4a98982e7dc3ed3ac6d4, user_id=72860d165fe246009c17df0e53800f50, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:38.463770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.539706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1206/2385] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:38 2022] PUT /identity/v3/projects/b40cfbfeef1b4a98982e7dc3ed3ac6d4/users/72860d165fe246009c17df0e53800f50/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:38.550194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.550499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.550773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.583142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] MFA Rules not processed for user `72860d165fe246009c17df0e53800f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:38.584380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:38.654444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1180/2386] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:38.665556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:38.667249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:38.728656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d27f-f600-4a90-91f3-62072a110937 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:38.732794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:38.733418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:38.733701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:38.733974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:38.743360 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:38.744824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890ae4-c4b3-41a3-9019-50225ab55f8e service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:38.754775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1207/2387] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:44.195928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:44.220746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:44.257392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7fdc6-5cfb-4b23-a589-6f122a43ed56 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:44.261847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:44.262480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:44.262756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:44.263025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] PATH_INFO: `/v3/users/72860d165fe246009c17df0e53800f50` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:44.264166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72860d165fe246009c17df0e53800f50)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:44.266132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:44.489767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72860d165fe246009c17df0e53800f50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:44.490514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72860d165fe246009c17df0e53800f50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:44.561475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72860d165fe246009c17df0e53800f50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:44.562101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72860d165fe246009c17df0e53800f50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:44.569840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72860d165fe246009c17df0e53800f50'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:44.587767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1181/2388] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:40:44 2022] DELETE /identity/v3/users/72860d165fe246009c17df0e53800f50 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:44.592324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:44.592812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:44.592908 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:44.611972 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:44.613209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:44.670239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1208/2389] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:44.680456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:44.681848 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:44.737082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d9a-9ca9-47a7-a8c5-bafac0123e19 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:44.740733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:44.741274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:44.741550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:44.741819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:44.751502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:44.752933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc43eb-a852-48b2-9971-8e1e96f2e463 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:44.762083 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1182/2390] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:45.431867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:45.437979 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:45.448672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162bbab-4d62-4d7c-be0e-bb5cd9fbf464 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:45.454806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:45.455606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:45.455779 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:45.456058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] PATH_INFO: `/v3/projects/b40cfbfeef1b4a98982e7dc3ed3ac6d4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:45.457383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40cfbfeef1b4a98982e7dc3ed3ac6d4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:45.459203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:45.476320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:45.594675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40cfbfeef1b4a98982e7dc3ed3ac6d4', 'request_id': 'req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:45.600847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40cfbfeef1b4a98982e7dc3ed3ac6d4', 'request_id': 'req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:45.687471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40cfbfeef1b4a98982e7dc3ed3ac6d4', 'request_id': 'req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:45.698725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] The token cache is being invalidate because project b40cfbfeef1b4a98982e7dc3ed3ac6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:40:45.699022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:45.699661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15e6b795-4ca9-4c66-b5a9-ede91a6e8eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:40:45.701215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1209/2391] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:40:45 2022] DELETE /identity/v3/projects/b40cfbfeef1b4a98982e7dc3ed3ac6d4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:45.706915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:45.707394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:45.707505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:45.727652 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:45.728970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:45.775924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:45.794669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1183/2392] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:45.803267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:45.803562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:45.803871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:45.823474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:45.824719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:45.832773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1210/2393] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:45.837111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:45.847180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991aa19-71da-483b-acba-4153b2fb279c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:45.851720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:45.852323 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:45.852634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:45.852910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:45.853604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:40:45.853910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:45.855507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:45.867409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1184/2394] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:40:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:40:45.871709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:45.881123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ba2c-02dd-4a99-9a5a-15d58ca873cc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:45.885912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:45.886464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:45.886746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:45.887018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:45.887725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:45.889654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:45.998089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1211/2395] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:40:45 2022] POST /identity/v3/projects => generated 366 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:46.002754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:46.012809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f284a4-d818-4ace-b33a-77a90d1e7fad admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:46.084087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:46.084766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.085024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.085300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.085969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:46.087878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:46.171686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1185/2396] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:40:46 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:40:46.176268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:46.186698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57db3b0-0ec1-43cc-95c0-99f12de3ad7a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:46.191955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:46.192670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.192954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.193228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.193922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:46.195529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:46.205385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1212/2397] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:46.209231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:46.219179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ce9f3-6dc1-43b5-bf56-200684f2d52b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:46.223779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:46.224376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.224686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.224960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.225646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:46.227254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943053f9-17fc-4575-8f12-6a52812b09aa admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:46.238444 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1186/2398] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:40:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:40:46.242491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:46.252614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302caae-4282-4552-989d-bab32f994960 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:46.257604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:46.258312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.258594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.258869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] PATH_INFO: `/v3/projects/edaef4c8da0845e08a43cef6fa7ef367/users/bb7cb0611b124cd2aab8b580fe0bb729/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.273891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edaef4c8da0845e08a43cef6fa7ef367, user_id=bb7cb0611b124cd2aab8b580fe0bb729, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:46.276559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:46.377976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1213/2399] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:40:46 2022] PUT /identity/v3/projects/edaef4c8da0845e08a43cef6fa7ef367/users/bb7cb0611b124cd2aab8b580fe0bb729/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:40:46.389485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.389777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.390054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.420731 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] MFA Rules not processed for user `bb7cb0611b124cd2aab8b580fe0bb729`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:40:46.421937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:46.490927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1187/2400] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:40:46.502744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:40:46.504458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:40:46.562947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41df87e-b3fa-4bf9-874f-ad286081bd17 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:40:46.566777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:40:46.567394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:40:46.567682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:40:46.567960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:40:46.576298 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:40:46.577747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7599-8f97-4667-8c26-c72dd4f0df56 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:40:46.586405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1214/2401] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:40:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:40:56.487892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:40:56.582523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:21.887977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:21.889477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:21.986170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7369b-c523-43a9-97e8-690f2b0597c8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:21.993746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:21.994680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:21.995140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:21.995519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:21.997914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:22.011334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:22.013057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:22.023901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1188/2402] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:28.190560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:28.191006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:28.191206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:28.209991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] MFA Rules not processed for user `4c6fffcc3c0d4c49848fa6c2f6b90130`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:28.211160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:28.266209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1215/2403] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:30.972839 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:30.974870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:31.056745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1e50-4ade-47aa-a28c-32e1cb18bd15 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:31.060438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:31.061053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:31.061333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:31.061604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:31.063406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:31.120670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:31.122092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:31.130973 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1189/2404] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:38.259495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:41.126861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:42.404795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:42.406526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:42.418844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2af8b-baf9-4aac-9555-ed6892b55a74 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:42.422351 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:42.423018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:42.423229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:42.423500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:42.425187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:42.489479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:42.491440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:42.501504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1216/2405] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:46.433966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:46.435638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:46.451001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab09da-0be3-4a61-8f8c-ad5fb87816c1 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:46.456071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:46.456720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:46.457003 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:46.457279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:46.459627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:46.477212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:46.478832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69b21-45f2-4126-b309-70675a9533d9 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:46.489773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1190/2406] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:41:46.626268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:46.638195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5ab73-8b3f-4e32-83a7-938b3921053c service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:46.644087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:46.644840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:46.645189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:46.645535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] PATH_INFO: `/v3/users/bb7cb0611b124cd2aab8b580fe0bb729` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:46.647352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7cb0611b124cd2aab8b580fe0bb729)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:46.649445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:46.951542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7cb0611b124cd2aab8b580fe0bb729'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:46.952435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7cb0611b124cd2aab8b580fe0bb729'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:46.965075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7cb0611b124cd2aab8b580fe0bb729'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:46.966156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7cb0611b124cd2aab8b580fe0bb729'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:46.990896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7cb0611b124cd2aab8b580fe0bb729'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:47.164147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1217/2407] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:41:46 2022] DELETE /identity/v3/users/bb7cb0611b124cd2aab8b580fe0bb729 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:47.169546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:47.169962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:47.170366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:47.192524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:47.193993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:47.274612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1191/2408] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:47.283683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:47.285321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:47.347898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480787d-7203-4c62-a9a4-1a970d6c6345 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:47.352793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:47.353381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:47.353672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:47.354310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:47.366841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:47.368718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f041902-e1d1-4178-b1f6-bb08fba65ea9 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:47.379377 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1218/2409] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:47.906745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:47.909601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:47.926813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3315d2-fac2-43c2-a5c4-cdc89b2a223c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:47.932447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:47.933087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:47.933368 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:47.933641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] PATH_INFO: `/v3/projects/edaef4c8da0845e08a43cef6fa7ef367` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:47.934833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edaef4c8da0845e08a43cef6fa7ef367)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:47.936980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.082654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edaef4c8da0845e08a43cef6fa7ef367', 'request_id': 'req-bebaf196-a9c9-413d-99d1-32f170ce7cb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:48.091765 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edaef4c8da0845e08a43cef6fa7ef367', 'request_id': 'req-bebaf196-a9c9-413d-99d1-32f170ce7cb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:48.097937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edaef4c8da0845e08a43cef6fa7ef367', 'request_id': 'req-bebaf196-a9c9-413d-99d1-32f170ce7cb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:48.261049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] The token cache is being invalidate because project edaef4c8da0845e08a43cef6fa7ef367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:41:48.261353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:48.262203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bebaf196-a9c9-413d-99d1-32f170ce7cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:48.263828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1192/2410] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:41:47 2022] DELETE /identity/v3/projects/edaef4c8da0845e08a43cef6fa7ef367 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:48.269533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce29b-1654-403b-9695-748390191958 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.269983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce29b-1654-403b-9695-748390191958 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.270075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce29b-1654-403b-9695-748390191958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.290452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d42ce29b-1654-403b-9695-748390191958 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:48.291684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d42ce29b-1654-403b-9695-748390191958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:48.295802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:48.357398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1219/2411] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:48.366158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.366484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.366770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.389473 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:48.390732 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:48.399346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1193/2412] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:48.403758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ce29b-1654-403b-9695-748390191958 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:48.414313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ce29b-1654-403b-9695-748390191958 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:48.419280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:48.419984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.420289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.420592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.421293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:41:48.421598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:48.423529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.438559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1220/2413] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:41:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:41:48.443535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:48.453207 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8581b6e-05e1-42e4-90c2-605c480862b0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:48.459254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:48.459884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.460168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.460476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.461155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:48.463198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.667854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1194/2414] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:41:48 2022] POST /identity/v3/projects => generated 382 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:41:48.671861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:48.682683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622df1d7-085b-4b8e-a33f-535cf795c25f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:48.738760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:48.739533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.740434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.740716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.741385 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:48.743282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.910888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1221/2415] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:41:48 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:41:48.916751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:48.930781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8f7b8-11e3-465d-aba9-45b7d1c6aae5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:48.935547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:48.936191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.936508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.936781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.937615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:48.939356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.952587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1195/2416] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:41:48 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:41:48.955570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:48.970154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f43d8-42bd-4ca1-b37e-193347336dfb admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:48.975556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:48.976173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:48.976493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:48.976760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:48.977686 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:48.981418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cd17c-8068-42bf-87f4-850e5e2a8a68 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:48.993210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1222/2417] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:41:48 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:41:48.999019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:49.012106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add3ba-a374-4431-bfc7-1076ba12dde5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:49.019852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:49.020582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:49.020932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:49.021209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] PATH_INFO: `/v3/projects/ba880373700044f08ba1adbdb416fb55/users/af63552b750f4160a257596e3e0f5669/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:49.041929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba880373700044f08ba1adbdb416fb55, user_id=af63552b750f4160a257596e3e0f5669, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:49.045815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:49.220147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1196/2418] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:41:48 2022] PUT /identity/v3/projects/ba880373700044f08ba1adbdb416fb55/users/af63552b750f4160a257596e3e0f5669/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:49.225383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab47850-e943-4c74-9602-d58f4717a94b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:49.225888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab47850-e943-4c74-9602-d58f4717a94b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:49.226261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab47850-e943-4c74-9602-d58f4717a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:49.246417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8ab47850-e943-4c74-9602-d58f4717a94b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:49.247774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8ab47850-e943-4c74-9602-d58f4717a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:49.321756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1223/2419] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:49.333405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:49.335367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:49.345828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:49.405688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0d3b0-39e3-483f-a0c2-236057fb08a1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:49.409644 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:49.410246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:49.410532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:49.410807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:49.422598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:49.424287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:49.435261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1197/2420] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:51.404582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:51.405871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:51.406194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:51.440001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] MFA Rules not processed for user `af63552b750f4160a257596e3e0f5669`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:51.441556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:51.450276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:51.516326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1224/2421] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:51.527408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:51.528925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:51.610234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff131-4cbd-48ae-a13c-224efdb57fc3 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:51.613928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:51.614630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:51.615422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:51.615696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:51.626130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:51.626490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:51.626579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:51.627989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:51.636855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1198/2422] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:51.642963 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c026-81f6-45b9-a66d-78e326cb549e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:51.647909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:51.648581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:51.648865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:51.649138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] PATH_INFO: `/v3/users/3b089f2cc22449329110fd3cfafecd53` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:51.650388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b089f2cc22449329110fd3cfafecd53)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:51.652918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:52.140934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b089f2cc22449329110fd3cfafecd53'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.141954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b089f2cc22449329110fd3cfafecd53'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.148525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b089f2cc22449329110fd3cfafecd53'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.149157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b089f2cc22449329110fd3cfafecd53'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.161050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b089f2cc22449329110fd3cfafecd53'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.270244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1225/2423] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:41:51 2022] DELETE /identity/v3/users/3b089f2cc22449329110fd3cfafecd53 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:41:52.641667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:52.659595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1adcfe-5eef-4724-9df6-2722a4937941 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:52.731519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:52.732141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:52.732447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:52.732722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] PATH_INFO: `/v3/projects/22e3bf7bb6ef4ca8bb2353bdfe4e90a3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:52.734377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e3bf7bb6ef4ca8bb2353bdfe4e90a3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:52.736355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:52.894536 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:52.900649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e3bf7bb6ef4ca8bb2353bdfe4e90a3', 'request_id': 'req-ed52bed6-ef42-4667-bd04-49806c9559e9'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.906998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e3bf7bb6ef4ca8bb2353bdfe4e90a3', 'request_id': 'req-ed52bed6-ef42-4667-bd04-49806c9559e9'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:52.912361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e3bf7bb6ef4ca8bb2353bdfe4e90a3', 'request_id': 'req-ed52bed6-ef42-4667-bd04-49806c9559e9'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.063786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] The token cache is being invalidate because project 22e3bf7bb6ef4ca8bb2353bdfe4e90a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:41:53.064078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.064714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.066255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1199/2424] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:41:52 2022] DELETE /identity/v3/projects/22e3bf7bb6ef4ca8bb2353bdfe4e90a3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:53.070827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:53.076223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:53.086109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71031e8-c551-45d8-ab9a-bbcc55d36920 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:53.140895 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:53.141543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:53.141831 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:53.142107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] PATH_INFO: `/v3/users/8573cfc4ecda47ab8480769977aeac07` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:53.143278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8573cfc4ecda47ab8480769977aeac07)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:53.145265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:53.518071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8573cfc4ecda47ab8480769977aeac07'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.518721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8573cfc4ecda47ab8480769977aeac07'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.525305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8573cfc4ecda47ab8480769977aeac07'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.525940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8573cfc4ecda47ab8480769977aeac07'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.537516 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-feab3017-cefd-4c3d-9d46-be3f38577788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8573cfc4ecda47ab8480769977aeac07'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:53.540828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:53.801742 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1226/2425] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:41:53 2022] DELETE /identity/v3/users/8573cfc4ecda47ab8480769977aeac07 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:41:54.140950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:54.142457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:54.158631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52bed6-ef42-4667-bd04-49806c9559e9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:54.216317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:54.217549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.217549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.217775 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] PATH_INFO: `/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.218743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4ae6c7e6f846dba2b27407c6a30d14)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:54.220674 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:54.386782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4ae6c7e6f846dba2b27407c6a30d14', 'request_id': 'req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:54.393024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4ae6c7e6f846dba2b27407c6a30d14', 'request_id': 'req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:54.398567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4ae6c7e6f846dba2b27407c6a30d14', 'request_id': 'req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:54.521527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] The token cache is being invalidate because project 1e4ae6c7e6f846dba2b27407c6a30d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:41:54.528803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:54.528803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-94d7bb1c-bb6f-4377-b0b5-99fce5f15273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:41:54.529269 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1200/2426] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:41:54 2022] DELETE /identity/v3/projects/1e4ae6c7e6f846dba2b27407c6a30d14 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:54.534436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.534913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.535145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.555477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:54.556775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:54.613526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1227/2427] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:54.621680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.621970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.622300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.642653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:54.643788 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:54.651622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1201/2428] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:54.663137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:54.673491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf52f0-b1b5-426a-b18f-5c51562f5efa None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:54.678055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:54.678655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.678943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.679204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.679944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:41:54.680816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:54.682404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:54.693600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1228/2429] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:41:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:41:54.697785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:54.706833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e3f7-235d-4bb6-9917-529b5770f047 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:54.711541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:54.712135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.712440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.712715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.713380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:54.715242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9170d-ae57-4891-b936-b2e8809259c2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:54.817514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1202/2430] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:41:54 2022] POST /identity/v3/projects => generated 378 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:41:54.821584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:54.830431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ca03-5860-4107-8ac9-157e8dc532b8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:54.867442 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.867714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.867993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.887390 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] MFA Rules not processed for user `af63552b750f4160a257596e3e0f5669`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:54.888863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:54.889568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:54.889850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:54.890007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:54.890121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:54.890780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:54.892806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:54.909604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:41:54.944787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1203/2431] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:41:55.030996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1229/2432] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:41:54 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:41:55.035082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:55.045675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3b73-2163-45e9-abab-02eb8e94db4a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:55.050212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:55.050821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:55.051103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:55.051407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:55.052075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:55.053709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:55.064586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1204/2433] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:41:55 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:41:55.068549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:55.079870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3938-ee8f-44af-b460-4366b45ae45e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:55.084904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:55.085562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:55.085843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:55.086203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:55.086916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:55.088569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fe5cd-f414-4d95-b890-7e41cd58c23f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:55.099948 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1230/2434] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:41:55 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:41:55.104547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:55.115158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004be90-bbfc-4c85-9edd-41e0b36fe7b4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:55.122240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:55.122240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:55.122240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:55.122240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] PATH_INFO: `/v3/projects/94684df47fe14e55af2e72ab5d2ddc48/users/ea4ccbb465094f079dcc4ca3bcac59cb/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:55.138277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94684df47fe14e55af2e72ab5d2ddc48, user_id=ea4ccbb465094f079dcc4ca3bcac59cb, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:55.141010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:55.312145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1205/2435] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:41:55 2022] PUT /identity/v3/projects/94684df47fe14e55af2e72ab5d2ddc48/users/ea4ccbb465094f079dcc4ca3bcac59cb/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:41:55.317302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a358393-66dd-4f93-b29a-4dfc645ded11 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:55.317588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a358393-66dd-4f93-b29a-4dfc645ded11 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:55.317865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a358393-66dd-4f93-b29a-4dfc645ded11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:55.337935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5a358393-66dd-4f93-b29a-4dfc645ded11 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:41:55.339312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5a358393-66dd-4f93-b29a-4dfc645ded11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:55.397511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1231/2436] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:41:55.407167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:41:55.408908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:41:55.465324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25aba-b25c-42e1-9c26-748fecfe5189 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:41:55.469014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:41:55.469631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:41:55.469920 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:41:55.470184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:41:55.482290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:41:55.483719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:41:55.498201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1206/2437] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:41:55.733936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:01.502311 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:01.502921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:01.502921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:01.542203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] MFA Rules not processed for user `ea4ccbb465094f079dcc4ca3bcac59cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:42:01.543990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:01.639162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1232/2438] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:42:01.644088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:01.660039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6770d4-24be-4bb1-97d6-fe02096187ff service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:01.669327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:01.671280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:01.671472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:01.671844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:01.672621 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:01.674958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:01.828238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1207/2439] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:42:01 2022] POST /identity/v3/projects => generated 378 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:42:01.834420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:01.845832 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3beb4c-d8ef-4b63-928b-16a268d38ae9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:01.958293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:01.958925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:01.959213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:01.959489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:01.960143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:01.962663 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:02.123135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1233/2440] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:42:01 2022] POST /identity/v3/users => generated 296 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:42:02.132331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:02.145520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2770-9a72-444d-ab43-3ad403439d59 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:02.151079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:02.151917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:02.152378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:02.153027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:02.153729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:02.155393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:02.167109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1208/2441] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:42:02 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:42:02.171601 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:02.182075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91423110-6511-4a75-9a71-12d1b8ba9620 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:02.187092 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:02.187767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:02.188055 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:02.188350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] PATH_INFO: `/v3/projects/36857008c0684ec9bc627668fc1cef7b/users/14ba8146f9414dbe9f2a20c32832a29b/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:02.205964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36857008c0684ec9bc627668fc1cef7b, user_id=14ba8146f9414dbe9f2a20c32832a29b, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:02.208735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:02.323644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:02.333773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1234/2442] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:42:02 2022] PUT /identity/v3/projects/36857008c0684ec9bc627668fc1cef7b/users/14ba8146f9414dbe9f2a20c32832a29b/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:42:02.334989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:02.346281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495655-0a21-41fd-aa94-3028a3de4a03 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:02.408575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:02.409204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:02.409485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:02.409753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:02.410472 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:02.412069 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8022-b795-4ab6-ac70-db1640d844fd admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:02.418184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:02.423958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1209/2443] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:42:02 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:42:02.428087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:02.439279 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83091f0-6e40-4129-9eb3-4c51f183b834 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:02.443938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:02.444630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:02.444911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:02.445185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] PATH_INFO: `/v3/projects/36857008c0684ec9bc627668fc1cef7b/users/14ba8146f9414dbe9f2a20c32832a29b/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:02.447904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36857008c0684ec9bc627668fc1cef7b, user_id=14ba8146f9414dbe9f2a20c32832a29b, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:02.450719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:02.461655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1235/2444] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:42:02 2022] PUT /identity/v3/projects/36857008c0684ec9bc627668fc1cef7b/users/14ba8146f9414dbe9f2a20c32832a29b/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:42:09.789767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:09.794894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:09.794894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:09.848453 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] MFA Rules not processed for user `14ba8146f9414dbe9f2a20c32832a29b`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:42:09.849771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:09.952068 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1210/2445] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:42:09.965736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:09.968075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:10.104654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61fe1a-35e1-4a68-b3b1-b9ae99401a2b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:10.110409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:10.111030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:10.111318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:10.111654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:10.177770 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:10.179462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:10.189801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1236/2446] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:42:19.948339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=34 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:20.185391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:33.795808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:33.850929 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e054484-70ce-40a2-bf5b-66ab44f801a0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:33.854706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:33.855320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:33.855603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:33.855875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:33.857650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:33.871418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:33.872868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc711b7-593c-4920-90be-cf84ba8d776c service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:33.884286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1211/2447] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:42:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:42:35.070082 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:35.071508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:35.083827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba215d7-6456-48e4-a557-6354c0dc68bd service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:35.087369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:35.087974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:35.088253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:35.088560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:35.090271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:35.103195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:35.104641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:35.115307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1237/2448] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:42:38.496831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89ae0-d508-42e3-8665-61be2397178d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:38.497317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89ae0-d508-42e3-8665-61be2397178d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:38.497508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89ae0-d508-42e3-8665-61be2397178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:38.527036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-85f89ae0-d508-42e3-8665-61be2397178d None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:42:38.528237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-85f89ae0-d508-42e3-8665-61be2397178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:38.582406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1212/2449] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:42:38.739414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:38.740980 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:38.820703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20e907-a527-4680-b8bc-c030d960996b service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:38.825871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:38.826757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:38.827038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:38.827307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:38.835304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:38.836843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:38.845418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1238/2450] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:42:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:42:48.575675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=55 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:48.841689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=38 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:42:57.100270 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:57.100763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:57.100884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:57.122670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] MFA Rules not processed for user `ea4ccbb465094f079dcc4ca3bcac59cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:42:57.123278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:57.143075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1213/2451] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:42:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:42:58.302569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:42:58.322157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693742-27aa-4944-a6bd-3c87c0f185e3 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:42:58.333379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:42:58.334161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:42:58.334541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:42:58.334917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:42:58.336890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:42:58.428222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:42:58.429783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:42:58.443749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1239/2452] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:02.791812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:02.793221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:02.823883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd376071-f064-406e-9d33-0d0115c3d11f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:02.834844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:02.835557 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:02.835844 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:02.836120 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] PATH_INFO: `/v3/users/1b9d76409f834b478981c0bbc0c3d4b6` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:02.851225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9d76409f834b478981c0bbc0c3d4b6)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:02.853224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:03.193597 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9d76409f834b478981c0bbc0c3d4b6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:03.194293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9d76409f834b478981c0bbc0c3d4b6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:03.287223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9d76409f834b478981c0bbc0c3d4b6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:03.288027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9d76409f834b478981c0bbc0c3d4b6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:03.296451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0cd0ef4-ab96-493b-bcfa-51f32e8936ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9d76409f834b478981c0bbc0c3d4b6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:03.330560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1214/2453] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:02 2022] DELETE /identity/v3/users/1b9d76409f834b478981c0bbc0c3d4b6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:03.822886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:03.834148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc744-292a-4a4a-b1ee-26c55736c96b service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:03.910571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:03.920310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:03.920310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:03.920310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] PATH_INFO: `/v3/projects/4088561009754a839604e3798787d8b9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:03.924905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4088561009754a839604e3798787d8b9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:03.926958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.070558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4088561009754a839604e3798787d8b9', 'request_id': 'req-22bc7450-92de-4c99-9346-6f23d04fdd02'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:04.077025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4088561009754a839604e3798787d8b9', 'request_id': 'req-22bc7450-92de-4c99-9346-6f23d04fdd02'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:04.167272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4088561009754a839604e3798787d8b9', 'request_id': 'req-22bc7450-92de-4c99-9346-6f23d04fdd02'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:04.180132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] The token cache is being invalidate because project 4088561009754a839604e3798787d8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:04.180542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:04.181320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-22bc7450-92de-4c99-9346-6f23d04fdd02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:04.182933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1240/2454] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:03 2022] DELETE /identity/v3/projects/4088561009754a839604e3798787d8b9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:04.188074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.189549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.189840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.213768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:04.215507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:04.292937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1215/2455] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:04.301564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.301853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.302271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.327249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:04.328861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:04.341765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1241/2456] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:04.346271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.356197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e78e0-330a-49bf-aecb-be43fad7ffc9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.361400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.362007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.362287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.362560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.363258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:43:04.363565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.365189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.377151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1216/2457] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:43:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:43:04.381853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.391946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7331-43b1-4d7e-a399-e12c010696d9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.397208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.397852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.398131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.398399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.399093 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.401853 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.407691 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:04.538262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1242/2458] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:04 2022] POST /identity/v3/projects => generated 364 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:04.543277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.564522 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873798f1-22e6-4773-9555-a9584132f0d9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.644552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:04.669337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.670031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.670317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.670609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.671280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.673819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.723575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.725466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:04.784010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1217/2459] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:04 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:04.791541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.805129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e804cfb-b222-4572-849b-75256ea05771 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.810290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.810901 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.811178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.811486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.812356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.814214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.820976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81a60a-3221-492d-bb50-ba86e42ba149 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.827574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.827574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.827574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.827574 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.829611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:04.840375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1218/2460] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:04 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:43:04.841182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.852172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902e840-8926-4da6-8a7d-aeeef64b7eeb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.859489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.860756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.861037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.861305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.862813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.863567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.877724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1219/2461] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:04 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:04.878931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:04.889258 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc38c-d21d-415e-be31-11d3ab104ab0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:04.894805 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:04.895863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:04.895863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:04.896079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] PATH_INFO: `/v3/projects/df64ffee96e34872aad7362c3be68fc2/users/bcbbd65047e54fe195b365924c0a8c60/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:04.913386 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df64ffee96e34872aad7362c3be68fc2, user_id=bcbbd65047e54fe195b365924c0a8c60, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.915609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:04.916122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.917076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44eb72-31e7-4e03-90bd-7a0bf36bae68 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:04.928332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1243/2462] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:04.990626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1220/2463] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:04 2022] PUT /identity/v3/projects/df64ffee96e34872aad7362c3be68fc2/users/bcbbd65047e54fe195b365924c0a8c60/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:05.002798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.003163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.003503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.033440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] MFA Rules not processed for user `bcbbd65047e54fe195b365924c0a8c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:05.035001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.112622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1244/2464] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:05.114293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.180771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204f493-fd26-4656-84c3-4e0175b7441b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.181732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.183843 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.186330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.187045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.187402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.187686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.188419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.190318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.197321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:05.253285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de17c4-076b-4644-8227-e8c78f6ca518 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.256993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.257664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.257949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.258223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.260023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.270499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.272125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.280399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1245/2465] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:05.301001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1221/2466] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:05 2022] POST /identity/v3/projects => generated 364 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:43:05.305460 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.317713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1b1e-bfd1-47da-9082-cd3242c61a05 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.377956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.378594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.379106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.379389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.380066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.382077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.473586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1246/2467] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:05 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:05.475603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.486384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b1fcc-6745-4244-9c5e-4edc65513c3d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.491446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.492132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.492437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.492715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.493436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.495463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.507420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1222/2468] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:05 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:05.511680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.523313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f9ced-3764-410b-8c4b-669bb8e3f9d6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.528482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.529236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.529592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.529941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] PATH_INFO: `/v3/projects/bcc518075d594c68b41a92839df3b61f/users/cae0e91150c64fd9bd591d6f6980d01f/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.547042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc518075d594c68b41a92839df3b61f, user_id=cae0e91150c64fd9bd591d6f6980d01f, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.549856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.654605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.711671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae4c7-526f-4d9e-b179-e6826e394446 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.715575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.716114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.716414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.716696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.718545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.726937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1247/2469] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:05 2022] PUT /identity/v3/projects/bcc518075d594c68b41a92839df3b61f/users/cae0e91150c64fd9bd591d6f6980d01f/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:05.728307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.733075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:05.795688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be15fb4-e01b-43dd-8b51-b507b074e823 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.798527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.799961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.801198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.801806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.802110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.802383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.803113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.805423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a36c-85ed-4272-bb10-96fbf192dcbe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.811860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1223/2470] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:05.816071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1248/2471] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:05 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:05.820152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.832365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a3169-9f2a-454a-9807-17e832ae76f4 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.837416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.838191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.838475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.838751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] PATH_INFO: `/v3/projects/bcc518075d594c68b41a92839df3b61f/users/cae0e91150c64fd9bd591d6f6980d01f/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.840855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc518075d594c68b41a92839df3b61f, user_id=cae0e91150c64fd9bd591d6f6980d01f, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:05.843519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:05.856812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1224/2472] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:05 2022] PUT /identity/v3/projects/bcc518075d594c68b41a92839df3b61f/users/cae0e91150c64fd9bd591d6f6980d01f/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:05.867650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.868184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.868184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.886455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:05.889746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.899258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:05.903038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] MFA Rules not processed for user `cae0e91150c64fd9bd591d6f6980d01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:05.904272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:05.973853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38978d4c-2a13-48b2-b021-f2aebb4040d4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:05.980517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:05.981147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1249/2473] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:05.981276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:05.981498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:05.981794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:05.983648 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:06.001152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:06.002735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:06.003383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:06.004904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:06.013306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1225/2474] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:06.062754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e5e2c-646f-4ead-b21a-eed2203cb129 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:06.067314 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:06.067935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:06.069398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:06.069398 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:06.078149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:06.079609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39246c1d-f68e-49ce-98f4-0bf0bfc4632b service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:06.090496 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1250/2475] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:07.100455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:07.174796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae688a-0649-4a0d-91b1-efd96a622288 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:07.179798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:07.180650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:07.180789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:07.181010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] PATH_INFO: `/v3/users/bcbbd65047e54fe195b365924c0a8c60` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:07.182215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbbd65047e54fe195b365924c0a8c60)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:07.184316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:07.213938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:07.435044 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbbd65047e54fe195b365924c0a8c60'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:07.435711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbbd65047e54fe195b365924c0a8c60'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:07.550551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbbd65047e54fe195b365924c0a8c60'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:07.551566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbbd65047e54fe195b365924c0a8c60'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:07.559171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbbd65047e54fe195b365924c0a8c60'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:07.576176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1226/2476] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:07 2022] DELETE /identity/v3/users/bcbbd65047e54fe195b365924c0a8c60 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:07.580656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:07.581195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:07.581290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:07.600431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:07.601641 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:07.666509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1251/2477] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:07.678171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:07.679636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:07.757288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50506a68-fad9-4e9d-a854-3bb388bb2c15 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:07.762724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:07.763463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:07.763913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:07.764282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:07.794131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:07.794131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78c606-dfb6-445a-8cea-7048c61520fb service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:07.814574 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1227/2478] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:08.598078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:08.599727 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:08.613877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf0e4-c0f8-4efb-81e6-9a5f8ac44675 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:08.618822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:08.619536 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:08.619822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:08.620102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] PATH_INFO: `/v3/projects/df64ffee96e34872aad7362c3be68fc2` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:08.621299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df64ffee96e34872aad7362c3be68fc2)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:08.623214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:08.778007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:08.787677 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df64ffee96e34872aad7362c3be68fc2', 'request_id': 'req-f881fe40-09db-48b5-8421-dba70a4dda88'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:08.795403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df64ffee96e34872aad7362c3be68fc2', 'request_id': 'req-f881fe40-09db-48b5-8421-dba70a4dda88'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:08.857960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:08.858452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:08.858551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:08.865297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df64ffee96e34872aad7362c3be68fc2', 'request_id': 'req-f881fe40-09db-48b5-8421-dba70a4dda88'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:08.876563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] The token cache is being invalidate because project df64ffee96e34872aad7362c3be68fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:08.882297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:08.882297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:08.882297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1252/2479] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:08 2022] DELETE /identity/v3/projects/df64ffee96e34872aad7362c3be68fc2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:08.883431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:08.884790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:08.911200 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:08.912913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:08.949058 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881fe40-09db-48b5-8421-dba70a4dda88 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:08.954388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:08.955012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:08.955290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:08.955557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] PATH_INFO: `/v3/users/cae0e91150c64fd9bd591d6f6980d01f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:08.956751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae0e91150c64fd9bd591d6f6980d01f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:08.958667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:08.978531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1228/2480] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:43:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:43:08.990502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:08.991855 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:09.005242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:09.080022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4c3a4-f4e0-49dd-a685-5579b51560b8 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:09.085130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:09.085748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:09.086031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:09.086303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:09.094169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:09.095739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:09.103681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1229/2481] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:43:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 16:43:09.232776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae0e91150c64fd9bd591d6f6980d01f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:09.233442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae0e91150c64fd9bd591d6f6980d01f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:09.240538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae0e91150c64fd9bd591d6f6980d01f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:09.241097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae0e91150c64fd9bd591d6f6980d01f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:09.261419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-25dc3ed9-281b-4892-9910-6a424947a397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae0e91150c64fd9bd591d6f6980d01f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:09.384637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1253/2482] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:08 2022] DELETE /identity/v3/users/cae0e91150c64fd9bd591d6f6980d01f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:09.883511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:09.895010 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849f76-9fff-4bf1-a13c-0f8ad2d61433 service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:09.950550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:09.951187 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:09.951467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:09.951744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] PATH_INFO: `/v3/projects/bcc518075d594c68b41a92839df3b61f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:09.952933 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc518075d594c68b41a92839df3b61f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:09.955463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.103963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc518075d594c68b41a92839df3b61f', 'request_id': 'req-ce5a5d26-18a8-4480-b178-b4861105e2b7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:10.110199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc518075d594c68b41a92839df3b61f', 'request_id': 'req-ce5a5d26-18a8-4480-b178-b4861105e2b7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:10.115568 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc518075d594c68b41a92839df3b61f', 'request_id': 'req-ce5a5d26-18a8-4480-b178-b4861105e2b7'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:10.206507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] The token cache is being invalidate because project bcc518075d594c68b41a92839df3b61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:10.206725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:10.207340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ce5a5d26-18a8-4480-b178-b4861105e2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:10.208883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1230/2483] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:09 2022] DELETE /identity/v3/projects/bcc518075d594c68b41a92839df3b61f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:10.213801 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.214288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.214388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.233565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:10.235480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:10.274573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:10.292828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1254/2484] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:10.301891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.302182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.302487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.322291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:10.323493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:10.331256 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1231/2485] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:10.335359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.345238 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbd578-01ac-4452-a588-d9a045a7269c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.354443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.354443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.354443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.354443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.354764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:43:10.354764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.354764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.365681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1255/2486] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:43:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:43:10.370288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.377112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:10.381783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4f898-698e-478e-bff3-ce6f0b055c1b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.386786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.387425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.387709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.387984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.388853 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.390803 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.511142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1232/2487] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:10 2022] POST /identity/v3/projects => generated 366 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:43:10.512353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.522750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935efa-1abe-4dd3-85a0-ca7f05a9fa9c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.580287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.580932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.581266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.581545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.582203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.584172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.733469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1256/2488] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:10 2022] POST /identity/v3/users => generated 284 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:10.738314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.749297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518b83-74f1-4aa3-82b8-face348786c3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.753820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.754428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.754708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.754975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.755669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.757342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.767783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1233/2489] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:10 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:43:10.771766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.782613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bac433-7b62-49d6-863b-e62686c281ef admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.787569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.788177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.788493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.788884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.789492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.791070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db61e9d-fc45-4744-9c43-ff5226a776a6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.803630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1257/2490] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:10 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:10.807897 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:10.818021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56989ec-9eb9-4f57-8465-4986a6816490 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:10.823059 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:10.828301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.828301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.828301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] PATH_INFO: `/v3/projects/14e606f778aa4c8c8b43a6aac60d2d1e/users/c3597ad4d4a044749ad68062eecbb64a/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.842699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e606f778aa4c8c8b43a6aac60d2d1e, user_id=c3597ad4d4a044749ad68062eecbb64a, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:10.845915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:10.921284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1234/2491] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:10 2022] PUT /identity/v3/projects/14e606f778aa4c8c8b43a6aac60d2d1e/users/c3597ad4d4a044749ad68062eecbb64a/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:10.931737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:10.932029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:10.932309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:10.959669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] MFA Rules not processed for user `c3597ad4d4a044749ad68062eecbb64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:10.960919 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:11.023512 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1258/2492] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:11.028061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.215205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe3380-31b7-4ab5-9a98-759f2ce7eed2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.219812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.220440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.220719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.220988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.221632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.223491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.322872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1235/2493] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:11 2022] POST /identity/v3/projects => generated 366 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:43:11.326892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.337635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49207e-cc1d-455c-8775-8527a9b36e2f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.375437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:11.392891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.393537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.393817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.394102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.394771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.397029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.493873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1259/2494] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:11 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:11.497873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.508261 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8590ac-5543-44f2-a1e7-c40a639f7c61 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.512907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.515762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.515762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.515762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.515762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.516275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.526466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1236/2495] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:11 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:11.530483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.541507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07add-32d0-4523-9404-f790b4585b7f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.546114 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.546845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.547127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.547973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] PATH_INFO: `/v3/projects/2e390d0e411b4d68890dae977ad2f651/users/acc4fcc041564c7abf928af1eb6da1c5/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.562720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e390d0e411b4d68890dae977ad2f651, user_id=acc4fcc041564c7abf928af1eb6da1c5, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.567374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.648695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1260/2496] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:11 2022] PUT /identity/v3/projects/2e390d0e411b4d68890dae977ad2f651/users/acc4fcc041564c7abf928af1eb6da1c5/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:11.656348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.667257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea1df-6a45-4294-83fc-cabea07b6642 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.726349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.726969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.727244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.727511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.728197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.729804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec352-c379-437f-905f-d2dd94ca1c3c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.740156 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1237/2497] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:11 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:11.744283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.754855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ee29b-d8c9-46da-860a-3bd22bdfa066 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.759559 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.760276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.760594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.760870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] PATH_INFO: `/v3/projects/2e390d0e411b4d68890dae977ad2f651/users/acc4fcc041564c7abf928af1eb6da1c5/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.762890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e390d0e411b4d68890dae977ad2f651, user_id=acc4fcc041564c7abf928af1eb6da1c5, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.765568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.776941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1261/2498] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:11 2022] PUT /identity/v3/projects/2e390d0e411b4d68890dae977ad2f651/users/acc4fcc041564c7abf928af1eb6da1c5/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:11.788158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.788477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.788746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.818097 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] MFA Rules not processed for user `acc4fcc041564c7abf928af1eb6da1c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:11.819319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:11.824870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:11.890019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1238/2499] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:11.901768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:11.903486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:11.962674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c1de5-69ca-44db-9750-e2fd3caf2787 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:11.966487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:11.967178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:11.967463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:11.967735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:11.977955 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:11.979397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:11.989160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1262/2500] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:12.157807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:12.167367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054b878-25db-4f61-8951-bcc8d04d5f4c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:12.170827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:12.171436 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:12.171714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:12.171986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:12.227333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:12.228793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d24f-effd-4063-8357-e95ef68a4902 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:12.237779 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1239/2501] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:43:15.926441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:15.987318 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928997-7e73-4df3-b149-c522382eae63 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:15.991926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:15.992597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:15.992788 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:15.993056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] PATH_INFO: `/v3/users/c3597ad4d4a044749ad68062eecbb64a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:15.994222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3597ad4d4a044749ad68062eecbb64a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:15.996164 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:16.219814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:16.227868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3597ad4d4a044749ad68062eecbb64a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:16.228555 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3597ad4d4a044749ad68062eecbb64a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:16.235215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3597ad4d4a044749ad68062eecbb64a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:16.235850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3597ad4d4a044749ad68062eecbb64a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:16.255643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3597ad4d4a044749ad68062eecbb64a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:16.351223 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1263/2502] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:15 2022] DELETE /identity/v3/users/c3597ad4d4a044749ad68062eecbb64a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:16.368925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:16.369448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:16.369534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:16.390039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:16.391804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:16.463287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1240/2503] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:16.473804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:16.476983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:16.599119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0148d9-4155-4a8d-b77a-cca40b504d24 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:16.603976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:16.604651 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:16.604923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:16.605455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:16.618747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:16.625466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:16.646014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1264/2504] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:17.168857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:17.170385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:17.184018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62168dd4-f0fd-405f-ad66-f7e25239ee84 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:17.189292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:17.189936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:17.190218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:17.190502 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] PATH_INFO: `/v3/projects/14e606f778aa4c8c8b43a6aac60d2d1e` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:17.191663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e606f778aa4c8c8b43a6aac60d2d1e)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:17.193602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:17.302430 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:17.310781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e606f778aa4c8c8b43a6aac60d2d1e', 'request_id': 'req-4474a521-1d23-4980-a53f-7f98bc7a82a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.316941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e606f778aa4c8c8b43a6aac60d2d1e', 'request_id': 'req-4474a521-1d23-4980-a53f-7f98bc7a82a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.322671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e606f778aa4c8c8b43a6aac60d2d1e', 'request_id': 'req-4474a521-1d23-4980-a53f-7f98bc7a82a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.435660 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] The token cache is being invalidate because project 14e606f778aa4c8c8b43a6aac60d2d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:17.436263 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.436539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.438086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1241/2505] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:17 2022] DELETE /identity/v3/projects/14e606f778aa4c8c8b43a6aac60d2d1e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:17.442102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:17.451685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31c268-1799-4f20-9ac0-c634b7b31ad3 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:17.507362 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:17.508007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:17.508290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:17.508597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] PATH_INFO: `/v3/users/acc4fcc041564c7abf928af1eb6da1c5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:17.509759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc4fcc041564c7abf928af1eb6da1c5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:17.511682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:17.783435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc4fcc041564c7abf928af1eb6da1c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.784174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc4fcc041564c7abf928af1eb6da1c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.793328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc4fcc041564c7abf928af1eb6da1c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.794245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc4fcc041564c7abf928af1eb6da1c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.796747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:17.810528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a8dc7508-b024-4863-9250-683511200143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc4fcc041564c7abf928af1eb6da1c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:17.911818 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1265/2506] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:17 2022] DELETE /identity/v3/users/acc4fcc041564c7abf928af1eb6da1c5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:18.428816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:18.432169 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:18.443324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474a521-1d23-4980-a53f-7f98bc7a82a6 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:18.519559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:18.520119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:18.520413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:18.520688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] PATH_INFO: `/v3/projects/2e390d0e411b4d68890dae977ad2f651` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:18.521852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e390d0e411b4d68890dae977ad2f651)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:18.523748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:18.653509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e390d0e411b4d68890dae977ad2f651', 'request_id': 'req-b49e6705-91e6-489d-9953-61031452ec95'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:18.662963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e390d0e411b4d68890dae977ad2f651', 'request_id': 'req-b49e6705-91e6-489d-9953-61031452ec95'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:18.674624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e390d0e411b4d68890dae977ad2f651', 'request_id': 'req-b49e6705-91e6-489d-9953-61031452ec95'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:18.796573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] The token cache is being invalidate because project 2e390d0e411b4d68890dae977ad2f651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:18.796573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:18.797459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b49e6705-91e6-489d-9953-61031452ec95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:18.799700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1242/2507] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:18 2022] DELETE /identity/v3/projects/2e390d0e411b4d68890dae977ad2f651 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:18.805661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:18.806187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:18.806320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:18.826957 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:18.828233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:18.932010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1266/2508] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:18.934587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:18.935125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:18.935603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:18.967088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:18.970817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:18.983042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1243/2509] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:18.988650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.003548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64774787-e599-4dc7-b208-bd78226f9b47 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.009230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.010167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.010645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.011119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.012157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:43:19.012687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.026513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.052586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1267/2510] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:43:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:43:19.057392 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.067944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3765dd9-e1be-446d-bda7-29251eec1944 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.079682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.080649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.081148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.081611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.082689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.085404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.200448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1244/2511] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:19 2022] POST /identity/v3/projects => generated 408 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:19.200664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.212250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424408a-200b-495f-b211-a0117454f782 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.314102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.314102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.314102 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.314525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.315067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.317076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.413847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1268/2512] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:19 2022] POST /identity/v3/users => generated 305 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:43:19.418112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.428746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab6807-e53f-4096-8a24-a6dcf43f0157 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.433217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.433812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.434082 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.434345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.435026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.436625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.447240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1245/2513] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:19 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:19.451241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.461187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8226e-5d56-4227-9844-0901ce63c560 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.465851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.466458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.466749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.467027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.467721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.469353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce158ce-9bc1-4406-b4d0-fa30b8e1d8fa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.479940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1269/2514] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:19 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:19.484057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.494626 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13637fae-eeaf-48ed-be62-10585f0d0afe admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.498966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.499619 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.499894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.500162 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] PATH_INFO: `/v3/projects/eabacaed686044c49511f58138bccd21/users/51f15864654c4a5cb1ea21c7e8c68234/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.512679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:19.515764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabacaed686044c49511f58138bccd21, user_id=51f15864654c4a5cb1ea21c7e8c68234, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.518567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.618622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1246/2515] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:19 2022] PUT /identity/v3/projects/eabacaed686044c49511f58138bccd21/users/51f15864654c4a5cb1ea21c7e8c68234/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:19.628722 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df75550-cc21-446d-94d4-886915959d8d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.629011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df75550-cc21-446d-94d4-886915959d8d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.629287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df75550-cc21-446d-94d4-886915959d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.656443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3df75550-cc21-446d-94d4-886915959d8d None None] MFA Rules not processed for user `51f15864654c4a5cb1ea21c7e8c68234`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:19.659072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3df75550-cc21-446d-94d4-886915959d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:19.694719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:19.729406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1270/2516] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:19.734012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.796699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709f706-7390-4a58-b6f7-a20b6bd6631c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.802239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.802845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.803124 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.803394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.804058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.806171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:19.914042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1247/2517] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:19 2022] POST /identity/v3/projects => generated 410 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:19.918013 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df75550-cc21-446d-94d4-886915959d8d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:19.928709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df75550-cc21-446d-94d4-886915959d8d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:19.988561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:19.989186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:19.989464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:19.989733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:19.990425 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:19.992595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.096920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1271/2518] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:19 2022] POST /identity/v3/users => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:20.098394 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:20.108971 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed76f-4d82-4f99-a969-d478990a9581 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:20.113662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:20.114282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.114624 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.114907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.115640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:20.117448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.133399 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1248/2519] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:20 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:20.134358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:20.149304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e77c-1488-4d2e-897c-cc775f784008 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:20.156027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:20.157759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.158226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.158918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] PATH_INFO: `/v3/projects/e225b10e44834e35b8cbb4614a7ea56a/users/6b7798736fe34dd7bff616e4db9c385b/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.176161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e225b10e44834e35b8cbb4614a7ea56a, user_id=6b7798736fe34dd7bff616e4db9c385b, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:20.178668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.273212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:20.278634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1272/2520] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:20 2022] PUT /identity/v3/projects/e225b10e44834e35b8cbb4614a7ea56a/users/6b7798736fe34dd7bff616e4db9c385b/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:20.306093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdae931-da37-4e7e-95c8-f7057b5c6a70 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:20.422492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:20.423283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.423671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.424070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.425123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:20.426905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b89c2-aa6d-41f6-92bc-f996b3cb0796 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.445681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1249/2521] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:20 2022] GET /identity/v3/roles => generated 1427 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:20.446690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:20.461732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c5fb-5d7c-4fde-a23f-150b9bb13097 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:20.474931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:20.475681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.476070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.476489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] PATH_INFO: `/v3/projects/e225b10e44834e35b8cbb4614a7ea56a/users/6b7798736fe34dd7bff616e4db9c385b/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.478637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e225b10e44834e35b8cbb4614a7ea56a, user_id=6b7798736fe34dd7bff616e4db9c385b, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:20.481719 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.503543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1273/2522] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:20 2022] PUT /identity/v3/projects/e225b10e44834e35b8cbb4614a7ea56a/users/6b7798736fe34dd7bff616e4db9c385b/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:20.514957 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37251c-4254-4795-9546-386636cc610b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.515432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37251c-4254-4795-9546-386636cc610b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.515764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37251c-4254-4795-9546-386636cc610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.552681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0e37251c-4254-4795-9546-386636cc610b None None] MFA Rules not processed for user `6b7798736fe34dd7bff616e4db9c385b`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:20.554014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0e37251c-4254-4795-9546-386636cc610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:20.606997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:20.631321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1250/2523] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:20 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:20.642607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:20.644422 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:20.714760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab8a9-34e3-4d86-ba4a-08fd3e394210 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:20.720169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:20.722065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:20.722065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:20.722065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:20.733237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:20.735489 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:20.751874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1274/2524] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:20 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:22.326054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37251c-4254-4795-9546-386636cc610b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:22.336653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37251c-4254-4795-9546-386636cc610b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:22.340842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:22.341525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:22.341811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:22.342081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:22.402098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:22.403535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:22.413698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1251/2525] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:22 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:30.743374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:32.410709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=34 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:40.567197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:40.568718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:40.626324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744479-1889-4779-8a57-395f4130c204 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:40.629886 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:40.630491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:40.630774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:40.631041 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:40.632835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:40.650372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:40.651857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fbed6-72d5-4c3d-97b0-1803dd41305a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:40.661449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1275/2526] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:41.603808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:41.671594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f2728-b1e7-46dc-b496-fac7dfc99c0d service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:41.676669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:41.677294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:41.677570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:41.677837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] PATH_INFO: `/v3/users/51f15864654c4a5cb1ea21c7e8c68234` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:41.678981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f15864654c4a5cb1ea21c7e8c68234)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:41.680905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:41.896857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f15864654c4a5cb1ea21c7e8c68234'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:41.897504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f15864654c4a5cb1ea21c7e8c68234'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:42.012835 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f15864654c4a5cb1ea21c7e8c68234'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:42.013591 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f15864654c4a5cb1ea21c7e8c68234'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:42.028235 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f15864654c4a5cb1ea21c7e8c68234'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:42.050767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1252/2527] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:41 2022] DELETE /identity/v3/users/51f15864654c4a5cb1ea21c7e8c68234 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:42.055493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7b62-a5df-4765-9388-05d9d31cb9c1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:42.056011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7b62-a5df-4765-9388-05d9d31cb9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:42.056119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7b62-a5df-4765-9388-05d9d31cb9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:42.075535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f33c7b62-a5df-4765-9388-05d9d31cb9c1 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:42.076805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f33c7b62-a5df-4765-9388-05d9d31cb9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:42.152403 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1276/2528] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:42.163036 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:42.164498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:42.227250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a8fed-5a0a-4b73-ae1b-3983a3890578 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:42.231054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:42.231676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:42.231955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:42.232222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:42.243138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:42.244668 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:42.254659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1253/2529] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:42.755237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:42.755629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:42.755798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:42.776420 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:42.777625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:42.834888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1277/2530] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:42.845610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:42.846216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:42.874062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:42.874892 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266233c-8a91-4d69-b3cc-f11663bc531f service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:42.875573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:42.880638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:42.881250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:42.881534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:42.881812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] PATH_INFO: `/v3/projects/eabacaed686044c49511f58138bccd21` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:42.883107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eabacaed686044c49511f58138bccd21)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:42.885065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:42.905664 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:42.931730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:42.952950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841af30-4cb2-428b-a4f4-1e56e324c6ba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:42.958564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:42.959178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:42.959456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:42.959751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:42.967804 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:42.969401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:42.977397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1278/2531] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:43:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 16:43:43.070592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabacaed686044c49511f58138bccd21', 'request_id': 'req-13131079-4c5c-47f2-a217-76769bfb3aa0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.080443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabacaed686044c49511f58138bccd21', 'request_id': 'req-13131079-4c5c-47f2-a217-76769bfb3aa0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.087168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabacaed686044c49511f58138bccd21', 'request_id': 'req-13131079-4c5c-47f2-a217-76769bfb3aa0'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.235110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] The token cache is being invalidate because project eabacaed686044c49511f58138bccd21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:43.236114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.236114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.237590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1254/2532] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:42 2022] DELETE /identity/v3/projects/eabacaed686044c49511f58138bccd21 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:43.241467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:43.251563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04649a-3664-4e4a-b085-1a800b4e253e service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:43.313180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:43.313822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:43.314104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:43.314380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] PATH_INFO: `/v3/users/6b7798736fe34dd7bff616e4db9c385b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:43.315568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7798736fe34dd7bff616e4db9c385b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:43.317562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:43.690914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7798736fe34dd7bff616e4db9c385b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.692069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7798736fe34dd7bff616e4db9c385b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.699336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7798736fe34dd7bff616e4db9c385b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.699997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7798736fe34dd7bff616e4db9c385b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.712706 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-858d9151-68ac-4f85-8c51-0f5a753c0d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7798736fe34dd7bff616e4db9c385b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:43.864585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1279/2533] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:43 2022] DELETE /identity/v3/users/6b7798736fe34dd7bff616e4db9c385b => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:44.482988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:44.482988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:44.500704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13131079-4c5c-47f2-a217-76769bfb3aa0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:44.584487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:44.586639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:44.586860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:44.587137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] PATH_INFO: `/v3/projects/e225b10e44834e35b8cbb4614a7ea56a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:44.589286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e225b10e44834e35b8cbb4614a7ea56a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:44.593282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:44.772769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e225b10e44834e35b8cbb4614a7ea56a', 'request_id': 'req-078428c9-de2f-4252-ba21-b535aa04f9ed'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:44.779052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e225b10e44834e35b8cbb4614a7ea56a', 'request_id': 'req-078428c9-de2f-4252-ba21-b535aa04f9ed'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:44.784807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e225b10e44834e35b8cbb4614a7ea56a', 'request_id': 'req-078428c9-de2f-4252-ba21-b535aa04f9ed'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:44.904218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] The token cache is being invalidate because project e225b10e44834e35b8cbb4614a7ea56a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:44.904538 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:44.905180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-078428c9-de2f-4252-ba21-b535aa04f9ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:44.906692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1255/2534] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:44 2022] DELETE /identity/v3/projects/e225b10e44834e35b8cbb4614a7ea56a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:44.912687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:44.915130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:44.915130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:44.933051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:44.934295 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:44.985664 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:44.995271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1280/2535] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:45.003168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.003555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.004459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.025061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:45.026302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:45.034650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1256/2536] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:45.040313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.050844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea571746-7543-49ad-98f0-65a7ff430eb7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.055505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.056134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.056474 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.056761 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.057543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:43:45.057851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.059522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.071976 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1281/2537] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:43:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:43:45.080415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.090179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144c9b-0efa-4573-93c0-74a213428c1a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.094868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.095504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.095781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.096048 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.096746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.098686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.248426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1257/2538] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:45 2022] POST /identity/v3/projects => generated 378 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:45.252504 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.263150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb868-fa91-4b97-a8f6-70da40f9ff06 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.318689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.319292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.319570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.319842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.320517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.322418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.438697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1282/2539] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:45 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:45.439963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.451339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df152b14-7dad-4969-aa8c-96b3230f00ac admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.456147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.456790 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.457066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.457342 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.458023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.459595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.460935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:45.473310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1258/2540] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:45 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:45.474797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.484965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f21e2f-f617-4b0b-9fa1-ff144e1b9b42 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.495085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.496044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.496175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.496479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.497187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.506768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd5065-3ff3-4292-83d8-2035a7dad0ee admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.512758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1283/2541] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:45 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:45.514093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.524346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cccc1-c9d7-4756-bd86-b0d42f31c69d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.529229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.529896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.530178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.530455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] PATH_INFO: `/v3/projects/d94b547e155f4fa8ba5a396d3dc0fe29/users/591c522ce79b4213928a948e4a1cbc01/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.545638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94b547e155f4fa8ba5a396d3dc0fe29, user_id=591c522ce79b4213928a948e4a1cbc01, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.548485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:45.795339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1259/2542] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:45 2022] PUT /identity/v3/projects/d94b547e155f4fa8ba5a396d3dc0fe29/users/591c522ce79b4213928a948e4a1cbc01/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:45.809543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.809829 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.810104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.837685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] MFA Rules not processed for user `591c522ce79b4213928a948e4a1cbc01`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:45.840148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:45.900575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1284/2543] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:45.904983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:45.968378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe12-6ab8-43a8-a097-7f2ed9f9c212 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:45.974065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:45.974688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:45.974969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:45.975240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:45.975921 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:45.977850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.136951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1260/2544] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:45 2022] POST /identity/v3/projects => generated 380 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:43:46.138078 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.149889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390c210-edb1-4bcb-b357-aa8889e03ffb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.198682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:46.208116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.208766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.209174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.209453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.210124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:46.212046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.361539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1285/2545] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:46 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:46.365400 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.375560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e44ab-cc83-418e-a727-e7287d1fafe4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.380498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.381109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.381388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.381662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.382357 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:46.383977 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.395146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1261/2546] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:46.398952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.409166 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b6733-6a43-48d5-8705-60700ff0dae0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.414563 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.415163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.415472 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.415746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.416461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:46.418046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef600-d737-46f0-ade2-0dbdb1997589 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.428035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1286/2547] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:46.432008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.442378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d859-5877-467d-b421-0fe5d2ab1043 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.446960 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.447604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.447887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.448164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] PATH_INFO: `/v3/projects/a0f14a1d22a141a191edd28ea005aa1d/users/17fd48adc2214abe81c9dfa23fe0ccb5/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.462896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f14a1d22a141a191edd28ea005aa1d, user_id=17fd48adc2214abe81c9dfa23fe0ccb5, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:46.465577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.586291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1262/2548] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:46 2022] PUT /identity/v3/projects/a0f14a1d22a141a191edd28ea005aa1d/users/17fd48adc2214abe81c9dfa23fe0ccb5/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:46.597814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.598097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.598369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.630655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] MFA Rules not processed for user `17fd48adc2214abe81c9dfa23fe0ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:46.631934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:46.709550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1287/2549] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:46.713987 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.785829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d5771-1c4e-454a-8373-48ffe44fae0c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.790587 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.791279 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.791467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.791728 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.792598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:46.794359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:46.923313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1263/2550] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:43:46 2022] POST /identity/v3/projects => generated 380 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:46.927759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:46.938481 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7566db-150f-4ae9-bf48-0fd1899d0e0e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:46.996534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:46.997162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:46.997447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:46.997724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:46.998390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.000315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.150258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1288/2551] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:43:46 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:43:47.154201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.165287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0591b8-6f49-424e-8644-12401948a7ae admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.169828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.170376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.170658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.171019 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.171628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.173246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.174530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:47.186065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1264/2552] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:47 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:47.190097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.200202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c95f-f24f-4c2e-8b98-7cc12490bca6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.204688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.205335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.205614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.205887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] PATH_INFO: `/v3/projects/1f0eea14126b458eb55256685746339f/users/34e1e6af86fe4c1db2e9e43b6e002a06/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.220255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0eea14126b458eb55256685746339f, user_id=34e1e6af86fe4c1db2e9e43b6e002a06, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.223438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.333613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1289/2553] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:47 2022] PUT /identity/v3/projects/1f0eea14126b458eb55256685746339f/users/34e1e6af86fe4c1db2e9e43b6e002a06/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:47.337658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.358999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc510c8-4254-4611-b5a5-8b08585dcd5f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.372098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.374609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:47.407652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:47.426653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.426653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.426653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.426653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.427144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.427144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.440870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1265/2554] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:43:47 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:43:47.441718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.451527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7276bea-4b54-488e-a7d5-215d8f743495 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.456512 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.457204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.457386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd64e9-7723-47e6-a33f-d0b3e94d58a9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.457519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.457756 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] PATH_INFO: `/v3/projects/1f0eea14126b458eb55256685746339f/users/34e1e6af86fe4c1db2e9e43b6e002a06/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.459747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0eea14126b458eb55256685746339f, user_id=34e1e6af86fe4c1db2e9e43b6e002a06, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.462740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88e27-c7fc-470e-8efe-30795e6bbcaf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.463915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.464568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.464857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.465141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.466931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:47.480437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.482035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.483810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1266/2555] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:43:47 2022] PUT /identity/v3/projects/1f0eea14126b458eb55256685746339f/users/34e1e6af86fe4c1db2e9e43b6e002a06/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:43:47.494670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.494954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.495223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.496087 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1290/2556] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:47.524027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] MFA Rules not processed for user `34e1e6af86fe4c1db2e9e43b6e002a06`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:47.525354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:47.593912 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1267/2557] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:43:47.605927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.607352 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:47.664795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722afe58-effb-4488-98ab-c29578189d5b service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.712769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.713303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.713584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.713856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.722881 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.724859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.734138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1291/2558] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:47.868221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:47.869634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:47.881683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76923-173a-41db-85a0-d31e7bf53a5d None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:47.885324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:47.885925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:47.886206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:47.891085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:47.942499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:47.943917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:47.952736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1268/2559] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:50.002007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:50.003440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:50.060060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6bbe-7f2a-48eb-8fdd-7700378f8750 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:50.063694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:50.064328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:50.064640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:50.064914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:50.066667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:50.095790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:50.136051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:50.137535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e04-d19b-48a0-a851-a3b3dfab2d70 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:50.147048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1292/2560] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:57.106343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:57.110525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:57.167457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5436-88bd-4d29-8ef1-cca2d7820895 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:57.172313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:57.172975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:57.173289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:57.173550 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] PATH_INFO: `/v3/users/591c522ce79b4213928a948e4a1cbc01` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:57.174692 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591c522ce79b4213928a948e4a1cbc01)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:57.177224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:57.414329 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591c522ce79b4213928a948e4a1cbc01'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:57.415008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591c522ce79b4213928a948e4a1cbc01'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:57.543999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591c522ce79b4213928a948e4a1cbc01'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:57.544676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591c522ce79b4213928a948e4a1cbc01'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:57.552183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591c522ce79b4213928a948e4a1cbc01'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:57.569046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1269/2561] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:57 2022] DELETE /identity/v3/users/591c522ce79b4213928a948e4a1cbc01 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:57.573669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227501-37c0-4246-b067-5375871b9be4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:57.574152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227501-37c0-4246-b067-5375871b9be4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:57.574275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227501-37c0-4246-b067-5375871b9be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:57.593301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-75227501-37c0-4246-b067-5375871b9be4 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:43:57.594503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75227501-37c0-4246-b067-5375871b9be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:57.656407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1293/2562] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:43:57.666630 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:57.668074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:57.729463 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a976bb-d682-4c22-acc1-697d22292df1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:57.733087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:57.733699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:57.733979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:57.734253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:57.743577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:57.745275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:57.758659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1270/2563] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:43:58.236495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75227501-37c0-4246-b067-5375871b9be4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:58.237981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75227501-37c0-4246-b067-5375871b9be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:58.255143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75227501-37c0-4246-b067-5375871b9be4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:58.261185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:58.261498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:58.261780 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:58.262405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] PATH_INFO: `/v3/projects/d94b547e155f4fa8ba5a396d3dc0fe29` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:58.263211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94b547e155f4fa8ba5a396d3dc0fe29)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:58.265165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:58.442564 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94b547e155f4fa8ba5a396d3dc0fe29', 'request_id': 'req-65196687-7ead-4733-ac6c-2805e38c9f72'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:58.449199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94b547e155f4fa8ba5a396d3dc0fe29', 'request_id': 'req-65196687-7ead-4733-ac6c-2805e38c9f72'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:58.669029 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94b547e155f4fa8ba5a396d3dc0fe29', 'request_id': 'req-65196687-7ead-4733-ac6c-2805e38c9f72'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:58.680367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] The token cache is being invalidate because project d94b547e155f4fa8ba5a396d3dc0fe29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:43:58.681752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:58.681752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:58.682994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1294/2564] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:58 2022] DELETE /identity/v3/projects/d94b547e155f4fa8ba5a396d3dc0fe29 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:58.686793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:58.696545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848315b-125d-4b1a-b4cf-ae59a5574b97 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:58.716588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:58.753363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:58.754024 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:58.754309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:58.754725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] PATH_INFO: `/v3/users/17fd48adc2214abe81c9dfa23fe0ccb5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:58.755917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fd48adc2214abe81c9dfa23fe0ccb5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:58.757900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:59.026289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fd48adc2214abe81c9dfa23fe0ccb5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.026861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fd48adc2214abe81c9dfa23fe0ccb5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.114175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fd48adc2214abe81c9dfa23fe0ccb5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.114819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fd48adc2214abe81c9dfa23fe0ccb5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.122122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fd48adc2214abe81c9dfa23fe0ccb5'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.140437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1271/2565] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:43:58 2022] DELETE /identity/v3/users/17fd48adc2214abe81c9dfa23fe0ccb5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:43:59.670848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:43:59.672297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:43:59.688364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65196687-7ead-4733-ac6c-2805e38c9f72 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:43:59.735702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:43:59.758654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:43:59.759302 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:43:59.759579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:43:59.759849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] PATH_INFO: `/v3/projects/a0f14a1d22a141a191edd28ea005aa1d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:43:59.761016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f14a1d22a141a191edd28ea005aa1d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:43:59.762914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:43:59.903131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f14a1d22a141a191edd28ea005aa1d', 'request_id': 'req-3b83f651-13eb-49bf-886f-7484d53b212d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:43:59.909525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f14a1d22a141a191edd28ea005aa1d', 'request_id': 'req-3b83f651-13eb-49bf-886f-7484d53b212d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.029589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f14a1d22a141a191edd28ea005aa1d', 'request_id': 'req-3b83f651-13eb-49bf-886f-7484d53b212d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.044529 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] The token cache is being invalidate because project a0f14a1d22a141a191edd28ea005aa1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:00.044840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.045482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.046998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1295/2566] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:43:59 2022] DELETE /identity/v3/projects/a0f14a1d22a141a191edd28ea005aa1d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:00.051125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:00.053158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:00.065815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128fa8-f640-4c81-8139-f8728e1060aa admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:00.124804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:00.125451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:00.125738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:00.126016 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] PATH_INFO: `/v3/users/34e1e6af86fe4c1db2e9e43b6e002a06` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:00.127180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e1e6af86fe4c1db2e9e43b6e002a06)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:00.129254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:00.312021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:00.313510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:00.345613 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:00.347199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e1e6af86fe4c1db2e9e43b6e002a06'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.348603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e1e6af86fe4c1db2e9e43b6e002a06'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.405534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83f651-13eb-49bf-886f-7484d53b212d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:00.412498 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:00.413115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:00.413394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:00.413665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:00.415390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:00.446712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e1e6af86fe4c1db2e9e43b6e002a06'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.447341 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e1e6af86fe4c1db2e9e43b6e002a06'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.454840 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e1e6af86fe4c1db2e9e43b6e002a06'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:00.473723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1272/2567] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:00 2022] DELETE /identity/v3/users/34e1e6af86fe4c1db2e9e43b6e002a06 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:00.498325 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:00.508819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:00.510720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091eabe-cbb4-45bd-957f-595fa6a06c4b service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:00.530533 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1296/2568] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:44:00.996734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:00.998178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:01.070927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4283c-4463-41f2-bebd-71785868da24 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:01.075423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:01.076050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.076327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.076629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] PATH_INFO: `/v3/projects/1f0eea14126b458eb55256685746339f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.077771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0eea14126b458eb55256685746339f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:01.079687 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:01.337142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0eea14126b458eb55256685746339f', 'request_id': 'req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:01.343384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0eea14126b458eb55256685746339f', 'request_id': 'req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:01.349025 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0eea14126b458eb55256685746339f', 'request_id': 'req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:01.504122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] The token cache is being invalidate because project 1f0eea14126b458eb55256685746339f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:01.504298 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:01.504909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-bdcb5a4b-0e94-4e5a-b93f-09dbe83359c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:01.506476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1273/2569] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:00 2022] DELETE /identity/v3/projects/1f0eea14126b458eb55256685746339f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:01.512556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.513033 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.513129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.536207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:01.537557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:01.597317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1297/2570] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:01.607500 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.607786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.608061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.628769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:01.629989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:01.638035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1274/2571] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:01.642251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:01.652099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7810edf-a47b-4061-a919-aa38db6d8580 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:01.656715 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:01.657321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.657605 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.657880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.658836 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:44:01.659139 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:01.661341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:01.673085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1298/2572] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:44:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:44:01.677966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:01.687396 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed7be7-5f1a-444e-9d59-67e7a7b0e8ba None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:01.692439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:01.693046 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.693347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.693641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.694313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:01.696179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:01.870953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1275/2573] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:01 2022] POST /identity/v3/projects => generated 372 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:01.875194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:01.885502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd83289-1221-4d0f-b2e4-6f8b4b4fbdaf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:01.942680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:01.943436 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:01.943793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:01.944154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:01.944848 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:01.947119 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:02.046665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1299/2574] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:01 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:02.051268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:02.061594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ab50-2f93-4db8-9427-72be620801a0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:02.066143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:02.067314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.067592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.067872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.068617 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:02.070197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:02.079996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1276/2575] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:02 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:02.087160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:02.098286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e58ce4-c410-474f-97ee-d01b4e2d8462 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:02.102975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:02.103590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.103875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.104148 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.104889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:02.106494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6771f27-fbc4-4076-9a60-d93d6e648679 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:02.118668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1300/2576] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:02 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:02.122934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:02.133572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b02fea-b99a-497c-ba12-4e10ac61c1e2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:02.138209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:02.138859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.139146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.139420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] PATH_INFO: `/v3/projects/1700205f1ea443f1854c725d43a15591/users/9995b19bf3cb45c987a92cfceab448ae/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.150248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:02.154151 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1700205f1ea443f1854c725d43a15591, user_id=9995b19bf3cb45c987a92cfceab448ae, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:02.157301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:02.298237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1277/2577] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:02 2022] PUT /identity/v3/projects/1700205f1ea443f1854c725d43a15591/users/9995b19bf3cb45c987a92cfceab448ae/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:02.368904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.369190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.369465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.397023 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] MFA Rules not processed for user `9995b19bf3cb45c987a92cfceab448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:02.402123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:02.437894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:02.463066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1301/2578] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:02.474919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:02.476702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:02.532640 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18437-6363-45f6-85a2-e5c7863ddedc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:02.537224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:02.537865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.538149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.538426 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.546866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:02.548354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da7587-f97a-4d88-9259-58e3200289fb service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:02.557628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1278/2579] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:02.579894 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:02.641479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a13cd-6e7a-4c6f-a323-29b9b075446e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:02.646458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:02.647096 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:02.647379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:02.647654 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] PATH_INFO: `/v3/users/9995b19bf3cb45c987a92cfceab448ae` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:02.648863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9995b19bf3cb45c987a92cfceab448ae)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:02.650768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:03.009181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9995b19bf3cb45c987a92cfceab448ae'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:03.009754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9995b19bf3cb45c987a92cfceab448ae'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:03.016759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9995b19bf3cb45c987a92cfceab448ae'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:03.017408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9995b19bf3cb45c987a92cfceab448ae'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:03.030183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9995b19bf3cb45c987a92cfceab448ae'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:03.127952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1302/2580] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:02 2022] DELETE /identity/v3/users/9995b19bf3cb45c987a92cfceab448ae => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:03.133184 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dba54-4101-4eba-b5e8-00ee4d25ab5f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:03.133913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dba54-4101-4eba-b5e8-00ee4d25ab5f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:03.134146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dba54-4101-4eba-b5e8-00ee4d25ab5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:03.153456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-868dba54-4101-4eba-b5e8-00ee4d25ab5f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:03.154655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-868dba54-4101-4eba-b5e8-00ee4d25ab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:03.212155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1279/2581] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:44:03.221814 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:03.223214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:03.279343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa252dc-19e5-4327-89cc-a45a715ae2ef admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:03.282837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:03.283699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:03.284014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:03.284276 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:03.293110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:03.294521 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:03.299313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:03.304371 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1303/2582] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:03.608203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:03.608525 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:03.608823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:03.627845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] MFA Rules not processed for user `af63552b750f4160a257596e3e0f5669`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:03.629723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:03.687485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1280/2583] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:03.962593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:03.964185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:03.975214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eca90-e7af-47a2-b04a-39aa9dafcd82 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:03.978864 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:03.979461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:03.979741 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:03.980007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:03.981698 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:03.990738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:03.992135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325230b5-296c-4dc2-a4fb-8cc8a3e28e49 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:04.000437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1304/2584] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:04.002567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:04.004631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:04.019462 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cef51-4427-4272-9da9-c41482b128fa None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:04.024883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:04.025520 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.025797 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.026072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] PATH_INFO: `/v3/projects/1700205f1ea443f1854c725d43a15591` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.027218 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1700205f1ea443f1854c725d43a15591)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:04.029283 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:04.189328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:04.201986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1700205f1ea443f1854c725d43a15591', 'request_id': 'req-92d2a694-1796-4dcc-b73f-3e623caeedb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:04.208276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1700205f1ea443f1854c725d43a15591', 'request_id': 'req-92d2a694-1796-4dcc-b73f-3e623caeedb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:04.213852 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1700205f1ea443f1854c725d43a15591', 'request_id': 'req-92d2a694-1796-4dcc-b73f-3e623caeedb8'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:04.406147 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] The token cache is being invalidate because project 1700205f1ea443f1854c725d43a15591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:04.406508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:04.407115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-92d2a694-1796-4dcc-b73f-3e623caeedb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:04.408916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1281/2585] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:04 2022] DELETE /identity/v3/projects/1700205f1ea443f1854c725d43a15591 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:04.422491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.422491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.422491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.437294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:04.438597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:04.527437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1305/2586] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:04.536072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.536225 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.536531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.563964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:04.565326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:04.577179 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1282/2587] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:04.578584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:04.589999 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ea7f6-dbb5-4d6f-a8fc-c92535c3c1b7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:04.595735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:04.596357 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.596669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.596950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.598205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:44:04.598556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:04.600177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:04.616842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1306/2588] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:44:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:44:04.617681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:04.627923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529496f-e721-4a98-841a-790b931d9b73 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:04.632655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:04.633265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.633547 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.633821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.634487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:04.636367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:04.820310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1283/2589] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:04 2022] POST /identity/v3/projects => generated 372 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:44:04.824443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:04.835572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5bf7-b44d-4ad4-b702-782050524037 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:04.891975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:04.892889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:04.892889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:04.893153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:04.893822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:04.896561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:05.046054 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1307/2590] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:04 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:05.050114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:05.060841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d65c-33e7-494f-a97b-0e2f985904f3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:05.065351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:05.065986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:05.066318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:05.067100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:05.067826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:05.069455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:05.079419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1284/2591] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:05.083766 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:05.094245 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c689-566e-401f-ac77-516a508a9826 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:05.098838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:05.099445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:05.099730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:05.100037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:05.100753 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:05.102471 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f85319-d848-48eb-9279-a0a2da8de68f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:05.112858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1308/2592] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:05.117470 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:05.130816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42993b7-9be0-4c3e-a048-2e5cae9dd6ce admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:05.136724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:05.137374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:05.137649 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:05.137916 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] PATH_INFO: `/v3/projects/9a3ead7dae63420bbffad38b5729179a/users/66c24eba17b2493f9e5e6b2416e7ba25/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:05.152566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3ead7dae63420bbffad38b5729179a, user_id=66c24eba17b2493f9e5e6b2416e7ba25, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:05.155167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:05.315904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1285/2593] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:05 2022] PUT /identity/v3/projects/9a3ead7dae63420bbffad38b5729179a/users/66c24eba17b2493f9e5e6b2416e7ba25/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:05.329487 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:05.329778 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:05.330060 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:05.361084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] MFA Rules not processed for user `66c24eba17b2493f9e5e6b2416e7ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:05.362389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:05.402623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:05.425572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1309/2594] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:05.437637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:05.439281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:05.496012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356cee1-0b12-40b5-9505-2287ab532a8c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:05.499781 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:05.500408 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:05.500708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:05.500985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:05.509362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:05.510778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:05.520841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1286/2595] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:15.515607 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:15.743652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:16.963388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:16.965030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:17.061661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d2786-ba81-440c-a8f1-03da43d28a91 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:17.071617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:17.071617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:17.071617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:17.071617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:17.071966 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:17.083648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:17.085256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:17.095602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1310/2596] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:27.091176 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=34 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:33.932554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:33.933975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:33.991449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67addfa9-a7d1-434e-b747-4eea8d34d565 service glance] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:33.994982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:33.995582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:33.995860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:33.996136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:33.997883 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:34.058511 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:34.059913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:34.069952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1287/2597] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:38.171133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:38.172666 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:38.187021 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657249d-62ff-4bbf-94b7-bfd8621c464e service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:38.191648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:38.192290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:38.192714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:38.193169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] PATH_INFO: `/v3/users/4c6fffcc3c0d4c49848fa6c2f6b90130` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:38.194353 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6fffcc3c0d4c49848fa6c2f6b90130)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:38.200995 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:38.465157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6fffcc3c0d4c49848fa6c2f6b90130'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:38.467817 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6fffcc3c0d4c49848fa6c2f6b90130'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:38.473475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6fffcc3c0d4c49848fa6c2f6b90130'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:38.474218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6fffcc3c0d4c49848fa6c2f6b90130'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:38.486112 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f3fce76c-b279-4f1f-86b8-6ac301554104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6fffcc3c0d4c49848fa6c2f6b90130'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:38.594417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1311/2598] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:38 2022] DELETE /identity/v3/users/4c6fffcc3c0d4c49848fa6c2f6b90130 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:38.933451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:38.944309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acee511-f36e-4874-8db5-f36eb4583bc5 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:39.005877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:39.006534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.006816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.007090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] PATH_INFO: `/v3/projects/8bb1d794693b410d8c759796d5e20281` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.008296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb1d794693b410d8c759796d5e20281)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:39.010266 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:39.184055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb1d794693b410d8c759796d5e20281', 'request_id': 'req-57d5cd58-3d44-4ad5-88eb-74921615c42c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:39.185925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:39.190381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb1d794693b410d8c759796d5e20281', 'request_id': 'req-57d5cd58-3d44-4ad5-88eb-74921615c42c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:39.196616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb1d794693b410d8c759796d5e20281', 'request_id': 'req-57d5cd58-3d44-4ad5-88eb-74921615c42c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:39.330482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] The token cache is being invalidate because project 8bb1d794693b410d8c759796d5e20281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:39.330768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:39.331388 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-57d5cd58-3d44-4ad5-88eb-74921615c42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:39.332936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1288/2599] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:38 2022] DELETE /identity/v3/projects/8bb1d794693b410d8c759796d5e20281 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:39.338968 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.339463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.339580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.358833 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:39.360036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:39.419731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1312/2600] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:39.428432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.428432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.428683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.452655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:39.453919 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:39.464592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1289/2601] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:39.477595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:39.486198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca36a1-15d0-42a8-a134-feba32149e61 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:39.496721 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:39.497768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.497768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.498123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.498900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:44:39.499299 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:39.501014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:39.515702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1313/2602] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:44:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:44:39.519778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:39.528947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dde5e-8cbf-4700-824e-aa54ba67be07 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:39.534219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:39.534847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.535141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.535421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.536091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:39.538084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:39.801823 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1290/2603] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:39 2022] POST /identity/v3/projects => generated 384 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:39.806128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:39.816731 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5ede1-c6f8-478e-beff-a55c09528662 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:39.871561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:39.872169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:39.872479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:39.872773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:39.873427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:39.875336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:39.883826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:40.079271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1314/2604] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:39 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:40.083467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.093891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55100e7a-30fb-4e2e-be4d-2a58911da616 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.099035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.099669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.099958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.100231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.100959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.102618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.113983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1291/2605] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:40.117969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.128106 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd4ad9-bc43-416c-bd63-d23c4bc43d46 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.134582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.135221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.135477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.135756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.136484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.140838 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289636e-d282-42b0-95d4-57844838eb19 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.151000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1315/2606] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:40 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:40.155084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.167493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb668-1144-4106-aa93-c27e75867d1f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.172524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.173189 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.173467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.177354 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] PATH_INFO: `/v3/projects/53492b96497c4a7d90f65844b0f58ff7/users/4c295fd29d7e48268d6de51a62c9a6f6/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.189417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=53492b96497c4a7d90f65844b0f58ff7, user_id=4c295fd29d7e48268d6de51a62c9a6f6, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.192643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.337103 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1292/2607] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:40 2022] PUT /identity/v3/projects/53492b96497c4a7d90f65844b0f58ff7/users/4c295fd29d7e48268d6de51a62c9a6f6/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:40.347297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.348338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.348500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.375594 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] MFA Rules not processed for user `4c295fd29d7e48268d6de51a62c9a6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:40.379091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:40.439178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1316/2608] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:40.443700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.506167 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf048eb-6ce2-467f-8063-72ee6a2417fd admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.511001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.511645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.511924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.512281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.513005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.514877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.670510 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1293/2609] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:40 2022] POST /identity/v3/projects => generated 384 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:40.674637 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.685922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b6b56-eb87-4e55-aa4f-1e4b8206f9d8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.746502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.753239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.753239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.753239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.753239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.753239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.954945 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1317/2610] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:40 2022] POST /identity/v3/users => generated 299 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:40.959469 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:40.971088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473fac-8fc2-4349-ba3c-856231362854 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:40.976190 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:40.976834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:40.977113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:40.977385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:40.978079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:40.979704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:40.990422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1294/2611] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:40.994841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:41.004709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02b27-5e9e-430f-925a-37c92b73535d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:41.009233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:41.009880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.010158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.010431 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] PATH_INFO: `/v3/projects/c27661aa80f64fa788bbd0b56786b22d/users/e9a55f2b9e1d47238c90ff2f95ae3fc1/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.025413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27661aa80f64fa788bbd0b56786b22d, user_id=e9a55f2b9e1d47238c90ff2f95ae3fc1, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:41.028180 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:41.159725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1318/2612] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:40 2022] PUT /identity/v3/projects/c27661aa80f64fa788bbd0b56786b22d/users/e9a55f2b9e1d47238c90ff2f95ae3fc1/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:41.163981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:41.173864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f7046-e8ff-4df5-aa26-80ee5aee76ad admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:41.223675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:41.233171 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:41.233784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.234066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.234337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.235021 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:41.236629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf847b-c44b-4240-a6b3-a8b76dba94a6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:41.250628 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1295/2613] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:41 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:41.255614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:41.265580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41238be-174f-468a-8fcb-c06df84a7cf9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:41.270211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:41.271432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.271713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.271990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] PATH_INFO: `/v3/projects/c27661aa80f64fa788bbd0b56786b22d/users/e9a55f2b9e1d47238c90ff2f95ae3fc1/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.274018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27661aa80f64fa788bbd0b56786b22d, user_id=e9a55f2b9e1d47238c90ff2f95ae3fc1, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:41.276670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:41.287266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1319/2614] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:41 2022] PUT /identity/v3/projects/c27661aa80f64fa788bbd0b56786b22d/users/e9a55f2b9e1d47238c90ff2f95ae3fc1/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:41.305608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.305898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.306176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.334237 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] MFA Rules not processed for user `e9a55f2b9e1d47238c90ff2f95ae3fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:41.335446 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:41.400078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1296/2615] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:41.412683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:41.414513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:41.437312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:41.481088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bd5d3-2bcd-43a0-9113-1f5b58cbde07 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:41.485921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:41.486600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.491459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.491459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.498065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:41.499554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:41.509635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1320/2616] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:41.843317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:41.853654 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48ec9c-89b6-48e9-9e74-75754c42d2f7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:41.858007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:41.858007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:41.858276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:41.858409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:41.917226 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:41.918735 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a2516-f084-4cdb-a930-632cf6d25fe5 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:41.928632 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1297/2617] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:42.580292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:42.644619 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad80f69-16fc-4804-aca3-0d3f60c46bd1 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:42.649796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:42.650428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:42.650805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:42.651000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] PATH_INFO: `/v3/users/4c295fd29d7e48268d6de51a62c9a6f6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:42.652129 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c295fd29d7e48268d6de51a62c9a6f6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:42.654057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:43.023816 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c295fd29d7e48268d6de51a62c9a6f6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:43.024830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c295fd29d7e48268d6de51a62c9a6f6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:43.031540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c295fd29d7e48268d6de51a62c9a6f6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:43.032160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c295fd29d7e48268d6de51a62c9a6f6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:43.044045 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c295fd29d7e48268d6de51a62c9a6f6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:43.137280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1321/2618] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:42 2022] DELETE /identity/v3/users/4c295fd29d7e48268d6de51a62c9a6f6 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:43.141882 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:43.142331 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:43.142429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:43.161647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:43.162838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:43.220751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1298/2619] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:43.230349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:43.231747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:43.279748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:43.291740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacb79-0d19-45c0-9d6e-bd891e0d6d16 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:43.295384 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:43.296003 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:43.296283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:43.296584 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:43.305943 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:43.307408 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:43.316670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1322/2620] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:43.869272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:43.870720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:43.882922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b493ba-6b26-4292-b472-657af80529ab None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:43.887562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:43.888230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:43.888543 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:43.888820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] PATH_INFO: `/v3/projects/53492b96497c4a7d90f65844b0f58ff7` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:43.890201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53492b96497c4a7d90f65844b0f58ff7)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:43.892081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:43.899402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:44.058004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53492b96497c4a7d90f65844b0f58ff7', 'request_id': 'req-38e14519-aab7-49fd-b147-3409febbd57d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.064360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53492b96497c4a7d90f65844b0f58ff7', 'request_id': 'req-38e14519-aab7-49fd-b147-3409febbd57d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.070706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53492b96497c4a7d90f65844b0f58ff7', 'request_id': 'req-38e14519-aab7-49fd-b147-3409febbd57d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.228443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] The token cache is being invalidate because project 53492b96497c4a7d90f65844b0f58ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:44.229086 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.229412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.230936 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1299/2621] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:43 2022] DELETE /identity/v3/projects/53492b96497c4a7d90f65844b0f58ff7 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:44.234993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:44.245505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0153d0-b423-4a65-b250-722bf7301395 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:44.300694 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:44.301328 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:44.301665 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:44.302514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] PATH_INFO: `/v3/users/e9a55f2b9e1d47238c90ff2f95ae3fc1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:44.303720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a55f2b9e1d47238c90ff2f95ae3fc1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:44.305669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:44.541421 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a55f2b9e1d47238c90ff2f95ae3fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.542072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a55f2b9e1d47238c90ff2f95ae3fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.548845 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a55f2b9e1d47238c90ff2f95ae3fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.549468 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a55f2b9e1d47238c90ff2f95ae3fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.561627 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9ce0110a-8c69-49ee-a01e-1e41ccedf15e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a55f2b9e1d47238c90ff2f95ae3fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:44.800484 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1323/2622] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:44 2022] DELETE /identity/v3/users/e9a55f2b9e1d47238c90ff2f95ae3fc1 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:45.344925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:45.349714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:45.359560 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e14519-aab7-49fd-b147-3409febbd57d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:45.416429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:45.417070 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:45.417379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:45.417655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] PATH_INFO: `/v3/projects/c27661aa80f64fa788bbd0b56786b22d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:45.418804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27661aa80f64fa788bbd0b56786b22d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:45.420816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:45.606586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27661aa80f64fa788bbd0b56786b22d', 'request_id': 'req-db90dbc2-9b03-41d7-b2c5-9b8296749980'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:45.612785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27661aa80f64fa788bbd0b56786b22d', 'request_id': 'req-db90dbc2-9b03-41d7-b2c5-9b8296749980'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:45.618858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27661aa80f64fa788bbd0b56786b22d', 'request_id': 'req-db90dbc2-9b03-41d7-b2c5-9b8296749980'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:45.787984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] The token cache is being invalidate because project c27661aa80f64fa788bbd0b56786b22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:45.788339 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:45.789004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-db90dbc2-9b03-41d7-b2c5-9b8296749980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:45.790573 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1300/2623] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:45 2022] DELETE /identity/v3/projects/c27661aa80f64fa788bbd0b56786b22d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:45.795575 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:45.796044 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:45.796150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:45.819771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:45.821009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:45.878854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1324/2624] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:45.886501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:45.886856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:45.887201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:45.907432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:45.908959 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:45.917288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1301/2625] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:44:45.921503 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:45.928128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:45.931931 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672094-5e1c-4791-8e8f-4d346bc2fc15 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:45.936531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:45.937134 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:45.937415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:45.937687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:45.938372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:44:45.938689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:45.940258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:45.951335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1325/2626] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:44:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:44:45.961275 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:45.970601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf363ff-cf57-46da-8df8-effbea090e2e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:45.975894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:45.976571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:45.976864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:45.977127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:45.977789 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:45.979678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:45.999941 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:46.379766 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1302/2627] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:45 2022] POST /identity/v3/projects => generated 384 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:46.385683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:46.396364 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04092-f42c-4122-8f0b-fa923842caef admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:46.452432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:46.453083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:46.453388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:46.453667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:46.454327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:46.456239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:46.656557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1326/2628] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:46 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:46.660940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:46.671385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac516c8-36c7-4f44-95b4-e10dfa98b594 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:46.676332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:46.676986 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:46.677251 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:46.677521 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:46.678209 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:46.679795 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:46.689831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1303/2629] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:46 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:46.694450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:46.704616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf9f79-cc81-4e76-b840-eba67cb67aa5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:46.709842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:46.710445 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:46.715039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:46.715039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:46.715039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:46.715039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7735e14-3d3f-46ed-9e22-52cfa5d62f07 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:46.724127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1327/2630] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:46.728211 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:46.738107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17c9e-ed1a-45cb-ac50-d53cdb1ff2e7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:46.743125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:46.743808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:46.744091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:46.744367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] PATH_INFO: `/v3/projects/75f56507ec874258b99c1083f5c9e555/users/8c3df9d6484540e5ba732e3cd269cc57/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:46.763322 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f56507ec874258b99c1083f5c9e555, user_id=8c3df9d6484540e5ba732e3cd269cc57, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:46.766000 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:46.896483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1304/2631] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:46 2022] PUT /identity/v3/projects/75f56507ec874258b99c1083f5c9e555/users/8c3df9d6484540e5ba732e3cd269cc57/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:46.908122 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:46.908883 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:46.908985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:46.937085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] MFA Rules not processed for user `8c3df9d6484540e5ba732e3cd269cc57`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:46.938297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:47.003822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1328/2632] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:47.008988 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.070828 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7259-fe33-445b-af1b-060f57ae3c73 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.075416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.076015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.076291 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.076593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.077363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.079223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.240381 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1305/2633] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:44:47 2022] POST /identity/v3/projects => generated 384 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:47.244343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.254934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b689f-72e7-4e2d-bc20-5b13c219a6a7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.310183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.310789 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.311073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.311376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.312035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.314002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.321764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:47.463207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1329/2634] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:44:47 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:44:47.467250 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.478629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02213-2a2e-49aa-be6a-befb7ff2ddf1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.483253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.483864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.484160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.484475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.485175 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.486760 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.497528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1306/2635] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:47 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:47.501825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.511927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c14a-f90e-4e92-b17e-16b981f6a1d1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.516609 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.517265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.517550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.517827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] PATH_INFO: `/v3/projects/4c41744f76264f07b7c86b21793e5e96/users/aad410619b3046b9b618d1271e44b41e/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.538200 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c41744f76264f07b7c86b21793e5e96, user_id=aad410619b3046b9b618d1271e44b41e, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.541067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.720203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1330/2636] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:47 2022] PUT /identity/v3/projects/4c41744f76264f07b7c86b21793e5e96/users/aad410619b3046b9b618d1271e44b41e/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:47.728705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.738554 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d76fa-6ff2-4cc6-ac9c-f7f9d48caaa3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.806807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.806807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.806807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.806807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.807293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.807293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56bf65-b4e7-4c52-9f04-8d06a6125059 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.807293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1307/2637] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:44:47 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:44:47.810782 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.821493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ba78d-e1bc-4a79-8cc7-139b7a68a675 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:47.826155 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:47.826846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.827128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.827400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] PATH_INFO: `/v3/projects/4c41744f76264f07b7c86b21793e5e96/users/aad410619b3046b9b618d1271e44b41e/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.829412 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c41744f76264f07b7c86b21793e5e96, user_id=aad410619b3046b9b618d1271e44b41e, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:47.832081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:47.842583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1331/2638] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:44:47 2022] PUT /identity/v3/projects/4c41744f76264f07b7c86b21793e5e96/users/aad410619b3046b9b618d1271e44b41e/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:47.852684 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:47.852969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:47.853238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:47.879682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] MFA Rules not processed for user `aad410619b3046b9b618d1271e44b41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:47.881487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:47.900305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:47.950417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1308/2639] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:44:47.961754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:47.963470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:48.143440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f509f-aa16-4c0d-9490-496bb03eee10 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:48.148117 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:48.148765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:48.149048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:48.149321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:48.203177 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:48.204621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6069b-dc6f-4904-a61e-aab5a98654bf service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:48.213940 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1332/2640] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:57.972114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:57.973553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:58.037152 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d648b4-1a60-49ff-ab99-3282e664856b None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:58.044818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:58.045461 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:58.045742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:58.046013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] PATH_INFO: `/v3/users/66c24eba17b2493f9e5e6b2416e7ba25` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:58.047230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c24eba17b2493f9e5e6b2416e7ba25)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:58.049130 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:58.209922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=5,keystone:SELECT=65 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:58.351358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c24eba17b2493f9e5e6b2416e7ba25'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:58.351944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c24eba17b2493f9e5e6b2416e7ba25'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:58.811221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c24eba17b2493f9e5e6b2416e7ba25'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:58.811851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c24eba17b2493f9e5e6b2416e7ba25'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:58.819438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c24eba17b2493f9e5e6b2416e7ba25'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:58.837172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1309/2641] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:44:57 2022] DELETE /identity/v3/users/66c24eba17b2493f9e5e6b2416e7ba25 => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:44:58.841596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:58.841882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:58.842156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:58.862572 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:44:58.863776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:58.927710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1333/2642] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:44:58.937592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:58.938999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:58.968056 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:58.969461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:58.982924 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:44:59.000138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b0333-c8ec-4982-b9d1-1eeecf9a50bc admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:59.004064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:59.004720 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:59.005054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:59.005337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:59.014456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:59.015886 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:59.025769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1310/2643] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:59.025928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463e7c5-521d-4eaa-aad1-4d4bac3c8cc3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:59.029525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:59.030140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:59.030415 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:59.030683 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:59.032488 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:59.098116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:59.099590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:59.110988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1334/2644] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:44:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:44:59.550181 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:44:59.550327 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:44:59.563618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d982e-160d-4335-a296-dd9c8fa09c1d service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:44:59.569239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:44:59.569872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:44:59.570150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:44:59.570423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] PATH_INFO: `/v3/projects/9a3ead7dae63420bbffad38b5729179a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:44:59.571614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3ead7dae63420bbffad38b5729179a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:44:59.574874 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:44:59.714825 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3ead7dae63420bbffad38b5729179a', 'request_id': 'req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:59.721026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3ead7dae63420bbffad38b5729179a', 'request_id': 'req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:59.726563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3ead7dae63420bbffad38b5729179a', 'request_id': 'req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:59.845011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] The token cache is being invalidate because project 9a3ead7dae63420bbffad38b5729179a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:44:59.845304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:59.846004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e39080f1-9ee6-4ab0-83d3-d6a8776e48dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:44:59.847527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1311/2645] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:44:59 2022] DELETE /identity/v3/projects/9a3ead7dae63420bbffad38b5729179a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:06.605847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:06.607309 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:06.621418 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caafd-4822-4318-8410-b2abecff0b87 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:06.673229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:06.684016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:06.684856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:06.685137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:06.685405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] PATH_INFO: `/v3/users/8c3df9d6484540e5ba732e3cd269cc57` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:06.686799 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3df9d6484540e5ba732e3cd269cc57)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:06.689222 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:06.968268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3df9d6484540e5ba732e3cd269cc57'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:06.968946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3df9d6484540e5ba732e3cd269cc57'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:06.976522 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3df9d6484540e5ba732e3cd269cc57'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:06.977165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3df9d6484540e5ba732e3cd269cc57'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:06.988765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3df9d6484540e5ba732e3cd269cc57'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:07.116214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1335/2646] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:45:06 2022] DELETE /identity/v3/users/8c3df9d6484540e5ba732e3cd269cc57 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:07.117651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c99487-0105-4852-acab-af8a62790821 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:07.118075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c99487-0105-4852-acab-af8a62790821 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:07.118219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c99487-0105-4852-acab-af8a62790821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:07.136905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-76c99487-0105-4852-acab-af8a62790821 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:07.138108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76c99487-0105-4852-acab-af8a62790821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:07.195935 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1312/2647] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:07.205758 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:07.207250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:07.263981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665fb6b-1ca8-44f1-827c-0153f559df82 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:07.267674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:07.268282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:07.268597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:07.268872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:07.278224 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:07.279644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:07.290131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1336/2648] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:07.840618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c99487-0105-4852-acab-af8a62790821 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:07.842280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76c99487-0105-4852-acab-af8a62790821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:07.855303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c99487-0105-4852-acab-af8a62790821 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:07.863863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:07.864850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:07.865774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:07.866194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] PATH_INFO: `/v3/projects/75f56507ec874258b99c1083f5c9e555` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:07.874770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f56507ec874258b99c1083f5c9e555)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:07.874770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:08.041145 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f56507ec874258b99c1083f5c9e555', 'request_id': 'req-2645e6d8-b07f-417f-be7d-ecc780328d2b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.047274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f56507ec874258b99c1083f5c9e555', 'request_id': 'req-2645e6d8-b07f-417f-be7d-ecc780328d2b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.049245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:08.053539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f56507ec874258b99c1083f5c9e555', 'request_id': 'req-2645e6d8-b07f-417f-be7d-ecc780328d2b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.175771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] The token cache is being invalidate because project 75f56507ec874258b99c1083f5c9e555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:45:08.176413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.182859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.182859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1313/2649] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:45:07 2022] DELETE /identity/v3/projects/75f56507ec874258b99c1083f5c9e555 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:08.183116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:08.192759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6167-2f94-43f9-bc0d-5c9c16177b02 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:08.263174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:08.264052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:08.264537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:08.264956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] PATH_INFO: `/v3/users/aad410619b3046b9b618d1271e44b41e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:08.266635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad410619b3046b9b618d1271e44b41e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:08.269653 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:08.286975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:08.590927 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad410619b3046b9b618d1271e44b41e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.591582 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad410619b3046b9b618d1271e44b41e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.599065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad410619b3046b9b618d1271e44b41e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.600303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad410619b3046b9b618d1271e44b41e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.612345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cca321b-0d3e-43ea-9b0c-679fb8133b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad410619b3046b9b618d1271e44b41e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:08.747025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1337/2650] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:45:08 2022] DELETE /identity/v3/users/aad410619b3046b9b618d1271e44b41e => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:09.315150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:09.319985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:09.332278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645e6d8-b07f-417f-be7d-ecc780328d2b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:09.389349 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:09.389989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:09.390271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:09.390546 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] PATH_INFO: `/v3/projects/4c41744f76264f07b7c86b21793e5e96` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:09.391691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c41744f76264f07b7c86b21793e5e96)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:09.393717 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:09.537669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c41744f76264f07b7c86b21793e5e96', 'request_id': 'req-17a1a470-ecef-4245-8818-a66cf379d2a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:09.544113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c41744f76264f07b7c86b21793e5e96', 'request_id': 'req-17a1a470-ecef-4245-8818-a66cf379d2a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:09.549777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c41744f76264f07b7c86b21793e5e96', 'request_id': 'req-17a1a470-ecef-4245-8818-a66cf379d2a6'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:09.684338 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] The token cache is being invalidate because project 4c41744f76264f07b7c86b21793e5e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:45:09.684662 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:09.685297 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-17a1a470-ecef-4245-8818-a66cf379d2a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:09.686850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1314/2651] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:45:09 2022] DELETE /identity/v3/projects/4c41744f76264f07b7c86b21793e5e96 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:09.692813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:09.694869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:09.694869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:09.712342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:09.713565 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:09.772320 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1338/2652] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:09.780922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:09.781210 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:09.781486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:09.803934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:09.805784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:09.814580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1315/2653] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:09.818860 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:09.828644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a671c-b9bf-4ece-bb65-09cfe3a59f9c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:09.833228 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:09.833821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:09.834101 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:09.834374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:09.835064 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:45:09.835369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:09.836987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:09.848625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1339/2654] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:45:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:45:09.853198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:09.862479 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e48d21-8f0a-4d34-8acd-afe7cb49899a None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:09.867771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:09.868361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:09.868665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:09.868932 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:09.869625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:09.871508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:09.994498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1316/2655] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:45:09 2022] POST /identity/v3/projects => generated 374 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:45:09.998681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:10.008613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa26918-992e-4e0b-a2fe-09684e1b0acd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:10.063748 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:10.064358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.064678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.064961 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.065612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:10.067495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:10.221182 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1340/2656] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:45:09 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:45:10.225878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:10.236071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2beff-30a8-4b07-8b27-0a43a2b88225 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:10.240730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:10.241367 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.241647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.241922 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.242618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:10.244214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:10.254418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1317/2657] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:10 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:45:10.269401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:10.279795 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8bd6f-5912-4249-8197-7a6c350b843c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:10.284678 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:10.285275 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.285553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.285826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.286502 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:10.288676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c944d5-8880-4d43-8faf-9a0f041b3544 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:10.290732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:10.299306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1341/2658] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:10 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:45:10.303246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:10.313281 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949759c-9f27-4f7e-b375-b825dd35304e admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:10.317887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:10.319112 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.319424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.319699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] PATH_INFO: `/v3/projects/319ddb5f712b4c8aa6fdaa01e62674f0/users/47c505966b4041788a1efee66df8444f/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.334709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=319ddb5f712b4c8aa6fdaa01e62674f0, user_id=47c505966b4041788a1efee66df8444f, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:10.337373 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:10.502565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1318/2659] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:45:10 2022] PUT /identity/v3/projects/319ddb5f712b4c8aa6fdaa01e62674f0/users/47c505966b4041788a1efee66df8444f/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:10.508152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82014005-a4fe-49e3-9463-ed81ba82f7a9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.508405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82014005-a4fe-49e3-9463-ed81ba82f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.508692 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82014005-a4fe-49e3-9463-ed81ba82f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.528049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-82014005-a4fe-49e3-9463-ed81ba82f7a9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:10.529283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-82014005-a4fe-49e3-9463-ed81ba82f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:10.586038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1342/2660] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:10.595871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:10.597590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:10.611080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:10.657170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58b385-8853-457f-ad32-d0e434e7f0ec admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:10.667334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:10.667334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:10.667334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:10.667334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:10.671361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:10.672827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:10.683109 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1319/2661] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:45:12.435537 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168f0-b346-4130-9fe5-dd50ac1c93c4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:12.436014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168f0-b346-4130-9fe5-dd50ac1c93c4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:12.436140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89168f0-b346-4130-9fe5-dd50ac1c93c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:12.463874 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d89168f0-b346-4130-9fe5-dd50ac1c93c4 None None] MFA Rules not processed for user `47c505966b4041788a1efee66df8444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:12.465181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d89168f0-b346-4130-9fe5-dd50ac1c93c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:12.530747 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1343/2662] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:12.542691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:12.544065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:12.600071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57c9e5-6565-49d7-9f91-f3211cef4e90 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:12.603740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:12.604352 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:12.604666 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:12.604942 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:12.613224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:12.614641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:12.624590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1320/2663] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:16.549600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:16.549600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:16.549600 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:16.565292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] MFA Rules not processed for user `47c505966b4041788a1efee66df8444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:16.566483 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:16.573467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1344/2664] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:18.007264 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:18.008713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:18.021703 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f481c-5efa-4a31-b603-91817ae2e9a8 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:18.025393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:18.026026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:18.026350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:18.026709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:18.028534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:18.110301 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:18.111749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:18.130149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1321/2665] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:26.560150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=63 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:28.127603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=88 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:33.227569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:33.229018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:33.241110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc597987-cafd-4c7f-a789-7f8a1a7f6b49 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:33.245025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:33.248254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:33.248254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:33.248254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:33.248254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:33.260929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:33.262343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:33.273520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1345/2666] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:37.770047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:37.772035 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:37.787113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8f3e6-af40-49b5-9957-a52a099671c0 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:37.791701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:37.792337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:37.792680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:37.792954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] PATH_INFO: `/v3/users/09de5ea60b9943adacc765b5ecdccb71` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:37.794117 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09de5ea60b9943adacc765b5ecdccb71)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:37.796027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:38.089348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09de5ea60b9943adacc765b5ecdccb71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.089946 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09de5ea60b9943adacc765b5ecdccb71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.370776 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09de5ea60b9943adacc765b5ecdccb71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.371421 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09de5ea60b9943adacc765b5ecdccb71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.379315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9d89aba8-2f47-46f1-8288-29bdc45c3a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09de5ea60b9943adacc765b5ecdccb71'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.397282 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1322/2667] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:45:37 2022] DELETE /identity/v3/users/09de5ea60b9943adacc765b5ecdccb71 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:38.752751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:38.760735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb31de-842b-4df8-9a3c-7a02882c9e25 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:38.819184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:38.824110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:38.824110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:38.824110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] PATH_INFO: `/v3/projects/f6a8f13fe22741d6843f193a0a959e57` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:38.824110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a8f13fe22741d6843f193a0a959e57)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:38.824110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:38.987278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a8f13fe22741d6843f193a0a959e57', 'request_id': 'req-8c4096a1-6797-4667-a7c0-9fe24926b4c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:38.993701 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a8f13fe22741d6843f193a0a959e57', 'request_id': 'req-8c4096a1-6797-4667-a7c0-9fe24926b4c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:39.100477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a8f13fe22741d6843f193a0a959e57', 'request_id': 'req-8c4096a1-6797-4667-a7c0-9fe24926b4c5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:39.111250 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] The token cache is being invalidate because project f6a8f13fe22741d6843f193a0a959e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:45:39.111538 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:39.112160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c4096a1-6797-4667-a7c0-9fe24926b4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:39.113699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1346/2668] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:45:38 2022] DELETE /identity/v3/projects/f6a8f13fe22741d6843f193a0a959e57 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:39.118818 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.119310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.119414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.139858 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:39.142746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:39.203780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1323/2669] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:39.213109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.217239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.217239 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.232854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:39.234036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:39.242475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1347/2670] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:39.246671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.256998 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da959e-af46-4c17-bb75-11db3dd68cf7 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.262138 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.262764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.263106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.263413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.264119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:45:39.264487 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.266234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.277989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1324/2671] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:45:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:45:39.282797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.302971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34aacc-b477-4154-8b41-04ebe15df1fd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.308765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.308765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.308992 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.309232 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.309921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.311862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.503597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1348/2672] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:45:39 2022] POST /identity/v3/projects => generated 374 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:45:39.512366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.518817 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fa26c-d9a3-44a3-ada0-866d94e2ec44 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.575693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.576260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.576586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.576867 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.577542 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.579733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.587537 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:39.714247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1325/2673] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:45:39 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:45:39.719142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.730978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feb9c1-8178-4416-b77c-8a75c1f36b89 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.735712 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.736324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.736635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.736907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.737597 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.739175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.752991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1349/2674] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:39 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:45:39.755829 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.765878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45931bd-4830-4069-9f6d-68d9c0fb34a7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.770519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.771143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.771428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.771702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.772405 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.774008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19abee4-6cba-46ef-9fb1-08ebcdc45030 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.784744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1326/2675] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:39 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:45:39.789338 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:39.799399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b787d-eb97-43fa-85c9-a863f07ef162 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:39.804844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:39.805541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.805822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.806099 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] PATH_INFO: `/v3/projects/be1b6060a485411fbc65d7f2b6a36376/users/adff29b8bea84e6495ed7475168df9bd/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:39.824289 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1b6060a485411fbc65d7f2b6a36376, user_id=adff29b8bea84e6495ed7475168df9bd, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:39.832255 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:39.973776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1350/2676] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:45:39 2022] PUT /identity/v3/projects/be1b6060a485411fbc65d7f2b6a36376/users/adff29b8bea84e6495ed7475168df9bd/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:39.983360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:39.983680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:39.983964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.014106 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] MFA Rules not processed for user `adff29b8bea84e6495ed7475168df9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:40.015375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:40.100947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1327/2677] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:40.105596 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.119072 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:40.169527 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662916b2-2ece-4a0a-98bc-3f852539a37c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.174462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.175027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.175305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.175578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.176287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.178449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.334018 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1351/2678] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:45:40 2022] POST /identity/v3/projects => generated 374 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:45:40.335894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.351484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad0f20-cdea-42d9-b1f6-d269ac5b4ad1 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.413017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.413637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.413915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.414315 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.414993 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.416944 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.660958 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1328/2679] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:45:40 2022] POST /identity/v3/users => generated 294 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:45:40.665034 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.675254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be608af-8898-4d12-b547-d1d81e97c0f2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.680330 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.681005 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.681270 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.681542 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.682252 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.683851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.695074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1352/2680] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:45:40.699013 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.709111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281471b-6456-426a-a006-5df1e0cc30c3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.713671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.714526 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.714810 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.715084 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] PATH_INFO: `/v3/projects/115467f1654f4293b0e6394fee0046e5/users/fd28117ecd8845388d4e68bce520c5ca/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.729454 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=115467f1654f4293b0e6394fee0046e5, user_id=fd28117ecd8845388d4e68bce520c5ca, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.732080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.872763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1329/2681] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:45:40 2022] PUT /identity/v3/projects/115467f1654f4293b0e6394fee0046e5/users/fd28117ecd8845388d4e68bce520c5ca/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:40.876746 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.887145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93523e6d-4abe-46fd-bb09-0dfabfb2b6dc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.941333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.941875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.942153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.942447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.943659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.945265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476601-638d-4acf-a305-b086ea05284a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.954671 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1353/2682] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:45:40 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:45:40.958980 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:40.969333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef6fa3-8d65-4878-b09c-62dc9d019cfa admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:40.973903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:40.974553 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:40.974836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:40.975113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] PATH_INFO: `/v3/projects/115467f1654f4293b0e6394fee0046e5/users/fd28117ecd8845388d4e68bce520c5ca/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:40.977163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115467f1654f4293b0e6394fee0046e5, user_id=fd28117ecd8845388d4e68bce520c5ca, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:40.979785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:40.991055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1330/2683] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:45:40 2022] PUT /identity/v3/projects/115467f1654f4293b0e6394fee0046e5/users/fd28117ecd8845388d4e68bce520c5ca/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:41.008211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:41.008616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:41.008922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:41.036661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] MFA Rules not processed for user `fd28117ecd8845388d4e68bce520c5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:41.037882 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:41.105284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1354/2684] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:41.116978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:41.118990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:41.141154 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:41.176739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c2178-1c9d-465b-9b29-0e2f79df80f5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:41.180976 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:41.181579 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:41.181888 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:41.182165 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:41.191105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:41.192693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:41.202102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1331/2685] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:41.443878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:41.447147 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:41.460159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb198ad-7713-4b56-8a30-82a3f6461605 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:41.466194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:41.466194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:41.466194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:41.466194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:41.482057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:41.544960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:41.546614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:41.559256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1355/2686] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:42.725793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:42.727276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:42.821050 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6a8d9-2ca6-49da-b917-9522a2a19183 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:42.826523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:42.827132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:42.827410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:42.827680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:42.829812 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:42.844182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:42.845777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:42.859307 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1332/2687] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:43.418417 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:43.419835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:43.492736 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800225a6-378f-43e3-8cb0-b3192b671500 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:43.497682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:43.498332 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:43.498614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:43.498889 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:43.500657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:43.512141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:43.513751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:43.524050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1356/2688] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:45:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:45:43.594198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:43.595851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:43.665758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c642b-dd5e-4415-9550-dd4c0ba3d498 service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:43.670532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:43.671185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:43.671468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:43.671742 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] PATH_INFO: `/v3/users/af63552b750f4160a257596e3e0f5669` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:43.672966 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af63552b750f4160a257596e3e0f5669)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:43.675014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:43.971719 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af63552b750f4160a257596e3e0f5669'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:43.972404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af63552b750f4160a257596e3e0f5669'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:43.975588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:44.073150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af63552b750f4160a257596e3e0f5669'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.073698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af63552b750f4160a257596e3e0f5669'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.081596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9b40d5bd-2ba8-4c39-8bc7-edaf439fcc51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af63552b750f4160a257596e3e0f5669'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.099278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1333/2689] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:45:43 2022] DELETE /identity/v3/users/af63552b750f4160a257596e3e0f5669 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:44.412486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:45:44.425085 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b8ff7-995e-4a3f-bb9a-2da4278c495d service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:45:44.495991 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:45:44.496640 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:44.496920 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:44.497199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] PATH_INFO: `/v3/projects/ba880373700044f08ba1adbdb416fb55` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:44.498397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba880373700044f08ba1adbdb416fb55)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:45:44.500287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:45:44.665161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:44.671720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba880373700044f08ba1adbdb416fb55', 'request_id': 'req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.678434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba880373700044f08ba1adbdb416fb55', 'request_id': 'req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.921211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba880373700044f08ba1adbdb416fb55', 'request_id': 'req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.929625 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] The token cache is being invalidate because project ba880373700044f08ba1adbdb416fb55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:45:44.929929 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.930560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3cce9b9b-aee9-4f37-9eb5-bdd6568ac0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:45:44.932240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1357/2690] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:45:44 2022] DELETE /identity/v3/projects/ba880373700044f08ba1adbdb416fb55 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:45:44.974931 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:45:44.985774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:45:44.985774 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:45:44.997498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] MFA Rules not processed for user `47c505966b4041788a1efee66df8444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:45:44.998738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:45:45.067030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1334/2691] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:45:55.060125 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:DELETE=3,keystone:SELECT=37 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:45:55.134444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:23.295237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf42-eed3-40b2-b2b9-7d304d6c93c0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:23.295822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf42-eed3-40b2-b2b9-7d304d6c93c0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:23.295822 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3eaf42-eed3-40b2-b2b9-7d304d6c93c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:23.316168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5d3eaf42-eed3-40b2-b2b9-7d304d6c93c0 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:23.317405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d3eaf42-eed3-40b2-b2b9-7d304d6c93c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:23.376140 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1358/2692] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:23.551539 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:23.558761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:23.621110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbb1d5-331d-47ba-9f2f-39e0b5456518 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:23.626075 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:23.626741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:23.626961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:23.627230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:23.636135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:23.637796 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:23.645691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1335/2693] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:46:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:46:33.369290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:33.643934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=32 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:40.591347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:40.599137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:40.599137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:40.610759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:40.612011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:40.619178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1359/2694] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:40.633667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:40.642489 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d9667-d801-4893-ba02-06d67c1dd19b service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:40.648027 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:40.648657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:40.648934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:40.649204 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:40.657185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:40.662164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:40.667017 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1336/2695] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:46:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:46:44.322145 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:44.323676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:44.393221 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567b4f-cf37-48b1-bb4f-72b3648f797c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:44.398313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:44.398973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:44.399256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:44.399531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] PATH_INFO: `/v3/users/47c505966b4041788a1efee66df8444f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:44.400737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c505966b4041788a1efee66df8444f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:44.402643 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:44.532589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:44.534047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:44.548739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e229f-6b70-4d61-98a0-6d4d1343ba77 service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:44.553311 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:44.553964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:44.554244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:44.554519 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] PATH_INFO: `/v3/users/adff29b8bea84e6495ed7475168df9bd` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:44.555679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adff29b8bea84e6495ed7475168df9bd)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:44.557616 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:44.657588 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c505966b4041788a1efee66df8444f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.658170 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c505966b4041788a1efee66df8444f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.665877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c505966b4041788a1efee66df8444f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.666510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c505966b4041788a1efee66df8444f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.678035 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-39c4af7a-2064-4e4f-a95c-df91d175cb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c505966b4041788a1efee66df8444f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.751904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1360/2696] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:46:44 2022] DELETE /identity/v3/users/47c505966b4041788a1efee66df8444f => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:44.785203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adff29b8bea84e6495ed7475168df9bd'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.785951 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adff29b8bea84e6495ed7475168df9bd'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.855041 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adff29b8bea84e6495ed7475168df9bd'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.855809 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adff29b8bea84e6495ed7475168df9bd'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.863387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adff29b8bea84e6495ed7475168df9bd'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:44.881847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1337/2697] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:46:44 2022] DELETE /identity/v3/users/adff29b8bea84e6495ed7475168df9bd => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:46:44.886514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:44.886865 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:44.887210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:44.906557 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:44.913435 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:44.966871 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1361/2698] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:44.976724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:44.978180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.009327 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.010949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.026263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec173f-24cd-457d-9c0c-cb82987567ee None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.031209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.031998 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.032286 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.032583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] PATH_INFO: `/v3/projects/319ddb5f712b4c8aa6fdaa01e62674f0` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.035006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=319ddb5f712b4c8aa6fdaa01e62674f0)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.037067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.040837 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f61c8-35b3-42ac-bfd7-2a98fcfdc2d9 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.045148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.045780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.046192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:45.046465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.046751 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.056247 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.057785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b726b-b8dd-4845-ac11-d9cb1d35da89 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.069001 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1338/2699] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:46:45.173634 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319ddb5f712b4c8aa6fdaa01e62674f0', 'request_id': 'req-b810135c-f2f3-4770-b251-787473e68d2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.180115 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319ddb5f712b4c8aa6fdaa01e62674f0', 'request_id': 'req-b810135c-f2f3-4770-b251-787473e68d2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.318862 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319ddb5f712b4c8aa6fdaa01e62674f0', 'request_id': 'req-b810135c-f2f3-4770-b251-787473e68d2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.330823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] The token cache is being invalidate because project 319ddb5f712b4c8aa6fdaa01e62674f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:46:45.331461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.331793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b810135c-f2f3-4770-b251-787473e68d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.333395 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1362/2700] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:46:45 2022] DELETE /identity/v3/projects/319ddb5f712b4c8aa6fdaa01e62674f0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:46:45.338545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.338832 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.339241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.360180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:45.361440 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.423403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1339/2701] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:46:45.431391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.431623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.431905 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.452247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:45.453523 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.461525 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1363/2702] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:45.465733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.470602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:45.476233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d2d52-a010-40dc-94db-c0828383a40f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.480891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.481497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.481780 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.482055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.482749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:46:45.483054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.484669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.497052 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1340/2703] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:46:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:46:45.501329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.511197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7fa53-1bce-45bd-b066-6e41c90b1301 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.515990 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.516631 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.516916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.517189 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.517850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.519791 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.556678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.556678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.563385 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc68ab-cb29-400a-aa59-c0bdd5ef7604 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.568271 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.568947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.569227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.569507 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] PATH_INFO: `/v3/projects/be1b6060a485411fbc65d7f2b6a36376` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.570685 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1b6060a485411fbc65d7f2b6a36376)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.572609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.617317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1364/2704] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:45 2022] POST /identity/v3/projects => generated 370 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:45.628317 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.681729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e950-1356-4cfd-adc5-5e393f32d215 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.686230 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.686826 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.687150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.687426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.688073 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.690009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.752963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1b6060a485411fbc65d7f2b6a36376', 'request_id': 'req-369eb33d-3218-44d7-80f0-15cd2bd1e10f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.759850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1b6060a485411fbc65d7f2b6a36376', 'request_id': 'req-369eb33d-3218-44d7-80f0-15cd2bd1e10f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.766393 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1b6060a485411fbc65d7f2b6a36376', 'request_id': 'req-369eb33d-3218-44d7-80f0-15cd2bd1e10f'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.856062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1365/2705] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:45 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:46:45.860022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.870169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106e25f-432d-4142-98b3-c2f57352ac7a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.896715 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] The token cache is being invalidate because project be1b6060a485411fbc65d7f2b6a36376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:46:45.897014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.898061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:45.899967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1341/2706] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:46:45 2022] DELETE /identity/v3/projects/be1b6060a485411fbc65d7f2b6a36376 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:46:45.907885 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.909267 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.922388 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:45.928676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.929300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.929583 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.929858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.930569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.932294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.942576 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1366/2707] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:45 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:46:45.946375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.948027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:45.959987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f16ed-afe6-4023-90ff-81f208838d46 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.964607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.965191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.965470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.965744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.966413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.967973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.972355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eb33d-3218-44d7-80f0-15cd2bd1e10f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.977080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:45.977714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:45.977997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:45.978277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] PATH_INFO: `/v3/users/fd28117ecd8845388d4e68bce520c5ca` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:45.978490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1367/2708] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:45 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:46:45.979459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd28117ecd8845388d4e68bce520c5ca)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:45.981417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:45.982163 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:45.992378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b443-123b-41c0-8740-46d37f0880d9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:45.999988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:46.000690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:46.000971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:46.001243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] PATH_INFO: `/v3/projects/5a64710ed3bf46fdae206b853de3ee2d/users/1c197746f76549e6923e961aceb9b1c3/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:46.020153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a64710ed3bf46fdae206b853de3ee2d, user_id=1c197746f76549e6923e961aceb9b1c3, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:46.022857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6010-57c1-433d-8dc9-f75042fbc4f1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:46.187263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1368/2709] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:45 2022] PUT /identity/v3/projects/5a64710ed3bf46fdae206b853de3ee2d/users/1c197746f76549e6923e961aceb9b1c3/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:46:46.228975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd28117ecd8845388d4e68bce520c5ca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:46.231952 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd28117ecd8845388d4e68bce520c5ca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:46.255133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:46.255416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:46.255688 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:46.283207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] MFA Rules not processed for user `1c197746f76549e6923e961aceb9b1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:46.284703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:46.327403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd28117ecd8845388d4e68bce520c5ca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:46.328144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd28117ecd8845388d4e68bce520c5ca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:46.335748 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd28117ecd8845388d4e68bce520c5ca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:46.343889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:46.352413 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1369/2710] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:46:46.354415 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1342/2711] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:46:45 2022] DELETE /identity/v3/users/fd28117ecd8845388d4e68bce520c5ca => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:46:46.359847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:46.361248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:46.441014 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3723f44-533c-48cc-9882-c5782a9670ca None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:46.481313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:46.489268 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:46.489879 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:46.490158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:46.490428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:46.545356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:46.546794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39773e4-907a-4d94-9990-63431c067287 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:46.556173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1370/2712] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:46:46.872328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:46.874243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:46.946665 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c637-6dfe-4672-a1a1-33f93ac84eb1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:46.952429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:46.953089 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:46.953371 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:46.953647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] PATH_INFO: `/v3/projects/115467f1654f4293b0e6394fee0046e5` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:46.955364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115467f1654f4293b0e6394fee0046e5)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:46.958212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.110348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115467f1654f4293b0e6394fee0046e5', 'request_id': 'req-e0d8b083-9db4-4f3d-be95-1276b0792ef4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:47.116725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115467f1654f4293b0e6394fee0046e5', 'request_id': 'req-e0d8b083-9db4-4f3d-be95-1276b0792ef4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:47.122441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115467f1654f4293b0e6394fee0046e5', 'request_id': 'req-e0d8b083-9db4-4f3d-be95-1276b0792ef4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:47.252294 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] The token cache is being invalidate because project 115467f1654f4293b0e6394fee0046e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:46:47.252534 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:47.253135 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-e0d8b083-9db4-4f3d-be95-1276b0792ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:47.255248 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1343/2713] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:46:46 2022] DELETE /identity/v3/projects/115467f1654f4293b0e6394fee0046e5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:47.260303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.260616 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.260888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.280649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:47.281847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:47.342057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1371/2714] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:47.350005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.350316 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.350608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.371155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:47.372343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:47.380864 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1344/2715] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:47.385125 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.395556 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfe13-f281-4afa-a758-7bc207ac6003 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.400205 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.400876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.401156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.401426 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.402113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:46:47.402579 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.404043 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.416150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1372/2716] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:46:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:46:47.426914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.436030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d00728-97e9-4891-9747-cb321ad7acee None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.440696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.441877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.442161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.442431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.443080 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.445002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.561791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1345/2717] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:47 2022] POST /identity/v3/projects => generated 388 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:47.570280 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.580803 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89b81-3777-429f-9352-fd2025e56279 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.636492 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.637097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.637374 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.637646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.638292 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.640193 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.752773 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1373/2718] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:47 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:47.757498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.767682 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407871c-90e8-4457-b35d-68febd07304f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.772212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.772873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.773157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.773431 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.774121 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.775710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.786746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1346/2719] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:47 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:47.792216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.802710 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb46805-ea84-4158-8ff1-1bcf261420fa admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.807242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.807867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.808146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.808444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.809130 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.810699 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40763e-9ed3-4604-b178-0740cf031543 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.821586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1374/2720] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:47 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:47.825876 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:47.835927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8824c-97c1-4687-9e93-34b6416ec03f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:47.840566 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:47.841272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.841551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.841826 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] PATH_INFO: `/v3/projects/1fe94a0152434323a1fc19ec2fd86150/users/a484868ce7694012bbd608b70b60ca2c/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.856964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe94a0152434323a1fc19ec2fd86150, user_id=a484868ce7694012bbd608b70b60ca2c, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:47.859636 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:47.952738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1347/2721] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:47 2022] PUT /identity/v3/projects/1fe94a0152434323a1fc19ec2fd86150/users/a484868ce7694012bbd608b70b60ca2c/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:47.963936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:47.964226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:47.964545 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:47.973076 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:47.993210 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] MFA Rules not processed for user `a484868ce7694012bbd608b70b60ca2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:47.994428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:48.058730 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1375/2722] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:48.063150 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.082074 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:48.129475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd998-2191-434b-82eb-39d6a1b47eb1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.134268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.135049 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.135423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.135702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.136614 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.138377 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.259448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1348/2723] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:48 2022] POST /identity/v3/projects => generated 388 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:48.260365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.271495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e5f8b-74ec-4891-ba00-3a0ca24b9f3e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.344011 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.344642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.344950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.346215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.346215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.348946 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.380483 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.382170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:48.457859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4ed4a-1009-4ec4-b048-b7276f0e624b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.462954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.463601 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.463878 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.464148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.465914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:48.478343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.479906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.489838 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1349/2724] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:46:48.505198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1376/2725] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:48 2022] POST /identity/v3/users => generated 301 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:48.510359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.521286 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed86f5a-3b77-4b4f-a5a6-2b499fd3225d service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.525979 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.526586 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.526861 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.527129 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.527816 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.529428 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.543676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1350/2726] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:48 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:46:48.544904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.555260 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf876d-bb82-4596-b9b6-af9385e7dd0f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.560184 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.560912 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.561219 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.561493 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] PATH_INFO: `/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b/users/7be9e9d892134fe3957a4d978ce0f39d/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.576792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500f3bd080a441c2b32bf1a89d00fd0b, user_id=7be9e9d892134fe3957a4d978ce0f39d, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.579669 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.746331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1377/2727] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:48 2022] PUT /identity/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b/users/7be9e9d892134fe3957a4d978ce0f39d/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:48.751420 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.763009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a4713-ac1d-4566-bc21-c9fd48f4232a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.832865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.833572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.833847 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.834119 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.834820 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.836484 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0304a-63f0-4839-9479-a08c57cc0e17 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.846846 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1351/2728] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:48 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:48.851116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:48.861186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe276-bd8c-4594-a1b3-b01fe83db033 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:48.866124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:48.866725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.867030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.867304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] PATH_INFO: `/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b/users/7be9e9d892134fe3957a4d978ce0f39d/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.869294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500f3bd080a441c2b32bf1a89d00fd0b, user_id=7be9e9d892134fe3957a4d978ce0f39d, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:48.871924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:48.882513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1378/2729] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:48 2022] PUT /identity/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b/users/7be9e9d892134fe3957a4d978ce0f39d/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:48.892671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:48.892969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:48.893253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:48.923634 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] MFA Rules not processed for user `7be9e9d892134fe3957a4d978ce0f39d`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:48.924896 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:48.930424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:48.993608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1352/2730] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:49.005455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:49.007168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:49.060157 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:49.065305 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b04818-3241-4428-b97b-92e7dd3dc297 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:49.069650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:49.070256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:49.070532 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:49.071749 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:49.079593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:49.081037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4ebd6-905e-40fa-a90e-7372ebd0be7b service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:49.090188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1379/2731] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:46:49.119493 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:49.181834 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845b9ba-42ee-4081-ae97-e9e4b4a39fa9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:49.186474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:49.187114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:49.187395 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:49.187669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] PATH_INFO: `/v3/users/a484868ce7694012bbd608b70b60ca2c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:49.188875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a484868ce7694012bbd608b70b60ca2c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:49.190784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:49.431062 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a484868ce7694012bbd608b70b60ca2c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:49.431620 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a484868ce7694012bbd608b70b60ca2c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:49.538159 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a484868ce7694012bbd608b70b60ca2c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:49.538794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a484868ce7694012bbd608b70b60ca2c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:49.547318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a484868ce7694012bbd608b70b60ca2c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:49.565432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1353/2732] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:46:49 2022] DELETE /identity/v3/users/a484868ce7694012bbd608b70b60ca2c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:49.569870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:49.570350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:49.570444 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:49.588880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:49.590088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:49.650566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1380/2733] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:49.663610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:49.665229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:49.692170 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:49.727215 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b2b52-a25c-4b9c-b71f-1fca303271bd admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:49.730729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:49.731336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:49.731612 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:49.731879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:49.741303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:49.742736 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:49.752078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1354/2734] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:46:50.313815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:50.315756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:50.329873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b881d6-3adf-4835-b885-212be9218b80 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:50.335191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:50.336355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:50.336861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:50.337143 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] PATH_INFO: `/v3/projects/1fe94a0152434323a1fc19ec2fd86150` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:50.338303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe94a0152434323a1fc19ec2fd86150)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:50.340811 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:50.490266 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe94a0152434323a1fc19ec2fd86150', 'request_id': 'req-9dc167a1-012b-43d9-84e3-476cf8be8977'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.496660 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe94a0152434323a1fc19ec2fd86150', 'request_id': 'req-9dc167a1-012b-43d9-84e3-476cf8be8977'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.592416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe94a0152434323a1fc19ec2fd86150', 'request_id': 'req-9dc167a1-012b-43d9-84e3-476cf8be8977'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.603844 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] The token cache is being invalidate because project 1fe94a0152434323a1fc19ec2fd86150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:46:50.604132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.604775 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.614208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1381/2735] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:46:50 2022] DELETE /identity/v3/projects/1fe94a0152434323a1fc19ec2fd86150 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:50.614347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:50.620577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9a6bc-a68f-420b-8e3e-6f9bfd17627c service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:50.678615 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:50.679288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:50.679571 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:50.679849 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] PATH_INFO: `/v3/users/7be9e9d892134fe3957a4d978ce0f39d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:50.681039 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be9e9d892134fe3957a4d978ce0f39d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:50.682961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:50.905506 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be9e9d892134fe3957a4d978ce0f39d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:50.906474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be9e9d892134fe3957a4d978ce0f39d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.015447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be9e9d892134fe3957a4d978ce0f39d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.016073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be9e9d892134fe3957a4d978ce0f39d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.023701 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-144e228c-0009-4081-abbc-585d9010d7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be9e9d892134fe3957a4d978ce0f39d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.045020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1355/2736] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:46:50 2022] DELETE /identity/v3/users/7be9e9d892134fe3957a4d978ce0f39d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:51.557449 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:51.563383 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:51.573002 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc167a1-012b-43d9-84e3-476cf8be8977 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:51.633020 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:51.633735 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:51.633915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:51.634181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] PATH_INFO: `/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:51.635323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=500f3bd080a441c2b32bf1a89d00fd0b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:51.637349 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:51.656590 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:51.816981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500f3bd080a441c2b32bf1a89d00fd0b', 'request_id': 'req-cd8385db-e8e4-4828-9515-357cea83f70d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.823048 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500f3bd080a441c2b32bf1a89d00fd0b', 'request_id': 'req-cd8385db-e8e4-4828-9515-357cea83f70d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.904196 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500f3bd080a441c2b32bf1a89d00fd0b', 'request_id': 'req-cd8385db-e8e4-4828-9515-357cea83f70d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.915517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] The token cache is being invalidate because project 500f3bd080a441c2b32bf1a89d00fd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:46:51.915805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.916462 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd8385db-e8e4-4828-9515-357cea83f70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:46:51.917989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1382/2737] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:46:51 2022] DELETE /identity/v3/projects/500f3bd080a441c2b32bf1a89d00fd0b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:51.922872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:51.923336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:51.923435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:51.944638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:51.945762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:51.985100 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:52.045697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1356/2738] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:52.053124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.053414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.053689 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.074511 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:52.075797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:52.084372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1383/2739] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:52.094336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.104938 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0df93b-a705-43a9-af0e-f2c64ba2428c None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.110058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.110686 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.110963 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.111232 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.111934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:46:52.112234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.113937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.125727 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1357/2740] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:46:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:46:52.130379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.139728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a04b8-f936-4b0f-9b27-ea43d7d4cb7d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.144852 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.146389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.146389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.146389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.147321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.148610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.282508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1384/2741] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:52 2022] POST /identity/v3/projects => generated 386 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:52.287188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.299197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c31156-02ef-4ab2-9f6f-1afc00ba1578 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.353819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.354448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.354730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.355023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.355710 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.357657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.493975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1358/2742] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:52 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:46:52.498057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.510359 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9ad-7cc7-4e65-b535-6258dbf56eac admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.514891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.515499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.515776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.516046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.516820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.518404 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.529427 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1385/2743] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:52 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:52.533422 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.544183 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fc73a-ef48-4eea-b6dd-77eae9f3f402 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.548821 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.549480 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.549704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.549978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.550672 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.552904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f5f0-8d57-4430-990a-cc8546b6d911 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.563374 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1359/2744] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:52 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:52.574365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.585419 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d9f9-33d8-430b-a1a8-e0e13988a5d4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.589960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.590618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.590896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.591167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] PATH_INFO: `/v3/projects/ae290ceecd3e44b1ad4809e9b18d2d01/users/badfc58fd6f74ce8894296a06f29b201/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.606137 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae290ceecd3e44b1ad4809e9b18d2d01, user_id=badfc58fd6f74ce8894296a06f29b201, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.610993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:52.733231 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1386/2745] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:52 2022] PUT /identity/v3/projects/ae290ceecd3e44b1ad4809e9b18d2d01/users/badfc58fd6f74ce8894296a06f29b201/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:52.799252 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.799606 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.799824 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.831242 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] MFA Rules not processed for user `badfc58fd6f74ce8894296a06f29b201`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:52.832488 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:52.897802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1360/2746] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:52.902458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:52.970214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9bd31-822d-4737-9634-45715e643a61 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:52.975373 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:52.975988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:52.976264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:52.976567 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:52.977229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:52.979132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.143592 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1387/2747] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:52 2022] POST /identity/v3/projects => generated 384 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:53.147053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.155702 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6dc6c-998b-46f8-a2b0-a3f1c1389475 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.195637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:53.216244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.216913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.217197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.217464 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.218163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.220274 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.351643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1361/2748] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:53 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:53.355441 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.366107 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8ad97-a525-4f82-b816-5e519bb14b0d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.370622 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.371261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.371539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.371809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.372668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.374308 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.386192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1388/2749] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:53 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:53.390234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.401102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500c4e-7ebf-4526-995d-aa6f08c623ca admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.405757 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.406361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.406642 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.406917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.407605 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.409255 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ea8ea-46ff-43ba-b34a-688481d17f30 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.420140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1362/2750] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:53.424296 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.434131 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1f9a-5b9b-4462-9f4b-1a4f5e929693 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.439214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.439872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.440159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.440447 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] PATH_INFO: `/v3/projects/db81b2f63e7b485f8161313fa6b06e4f/users/a899d5319e904697903668fe203ef50c/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.454215 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:53.458743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db81b2f63e7b485f8161313fa6b06e4f, user_id=a899d5319e904697903668fe203ef50c, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.461401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.581534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1389/2751] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:53 2022] PUT /identity/v3/projects/db81b2f63e7b485f8161313fa6b06e4f/users/a899d5319e904697903668fe203ef50c/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:53.592569 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.592865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.593148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.621974 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] MFA Rules not processed for user `a899d5319e904697903668fe203ef50c`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:53.623694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:53.691978 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1363/2752] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:46:53.697241 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.759896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949ef0-3cc8-4603-a1d9-33d9dfd0f554 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.764621 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.765227 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.765505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.765777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.766423 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.768300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:53.902065 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1390/2753] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:46:53 2022] POST /identity/v3/projects => generated 384 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:46:53.906451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:53.917503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6836285-2bd1-4e8d-87f3-c4af6f09f1ae None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:53.975037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:53.975577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:53.975862 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:53.976137 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:53.976854 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:53.978769 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.141739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1364/2754] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:46:53 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:46:54.146513 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:54.157128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8cbc4-6348-41d6-a2de-b77cead6f2ba admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:54.163764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:54.164820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.165121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.165400 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.167183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:54.168847 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.180680 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1391/2755] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:54 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:54.185296 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:54.197158 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85b63-952d-428d-8798-0352cc8023a1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:54.202092 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:54.202770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.203057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.203337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] PATH_INFO: `/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1/users/331b058bd97f4a38bb6f13a30efca185/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.218452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f72f058b9d4ef0897e28bf0b0774d1, user_id=331b058bd97f4a38bb6f13a30efca185, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:54.221143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.322365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1365/2756] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:54 2022] PUT /identity/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1/users/331b058bd97f4a38bb6f13a30efca185/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:54.326285 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:54.336075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd488e1-4e25-4507-8185-d352d3c6e66d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:54.390659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:54.391271 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.391548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.391841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.392551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:54.394136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31451929-2ef1-4d83-9844-675d8b9f3eec admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.403473 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1392/2757] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:46:54 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:46:54.407706 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:54.418509 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84346ea-8684-4dea-bfee-7ec330d86927 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:54.423439 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:54.424134 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.424441 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.424726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] PATH_INFO: `/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1/users/331b058bd97f4a38bb6f13a30efca185/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.427457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f72f058b9d4ef0897e28bf0b0774d1, user_id=331b058bd97f4a38bb6f13a30efca185, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:54.430223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.437099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:54.441663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1366/2758] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:46:54 2022] PUT /identity/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1/users/331b058bd97f4a38bb6f13a30efca185/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:46:54.453548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.453842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.458442 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.482656 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] MFA Rules not processed for user `331b058bd97f4a38bb6f13a30efca185`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:46:54.483876 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:54.530760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:46:54.554284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1393/2759] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:46:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:46:54.565203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:54.566961 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:54.627140 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d7747-08d0-4810-a0c6-e59eb422bcb5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:54.630877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:54.631485 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:54.631762 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:54.632030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:54.640904 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:54.642326 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:54.651618 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1367/2760] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:46:55.886802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:46:55.888569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:46:55.900837 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689022db-fd57-423a-b2aa-d5303badbfa6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:46:55.904459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:46:55.905074 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:46:55.905356 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:46:55.906835 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:46:55.973406 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:46:55.975530 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:46:55.985030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1394/2761] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:46:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:47:02.479552 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:02.488343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaded49-c212-4ec1-914f-e1997500a8b6 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:02.492132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:02.492778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:02.493064 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:02.493994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:02.550057 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:02.551468 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539161a6-e2f8-4a02-b025-0131f2752a71 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:02.560347 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1368/2762] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:05.980630 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=35 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:07.782921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:07.847263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc31fc-3f9f-45e4-a300-06f5efd27d3b service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:07.851840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:07.852510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:07.852793 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:07.853071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] PATH_INFO: `/v3/users/badfc58fd6f74ce8894296a06f29b201` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:07.854226 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badfc58fd6f74ce8894296a06f29b201)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:07.856128 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:08.213038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badfc58fd6f74ce8894296a06f29b201'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:08.213952 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badfc58fd6f74ce8894296a06f29b201'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:08.221469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badfc58fd6f74ce8894296a06f29b201'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:08.222098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badfc58fd6f74ce8894296a06f29b201'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:08.234550 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badfc58fd6f74ce8894296a06f29b201'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:08.363573 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1395/2763] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:07 2022] DELETE /identity/v3/users/badfc58fd6f74ce8894296a06f29b201 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:08.368747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:08.369278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:08.369278 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:08.392856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:08.394524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:08.465860 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1369/2764] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:08.474251 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:08.475662 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:08.535135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477f4d-85d4-4a04-9ff5-b70b495449f4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:08.538745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:08.539409 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:08.539806 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:08.540083 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:08.549195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:08.550599 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb45b1-0e19-48ef-bab7-aa2a6958f64b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:08.560534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1396/2765] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:09.253085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:09.253085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:09.266633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9d628-a558-47f4-b479-52cbf9aecc00 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:09.272295 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:09.272915 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:09.273195 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:09.273603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] PATH_INFO: `/v3/projects/ae290ceecd3e44b1ad4809e9b18d2d01` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:09.274801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae290ceecd3e44b1ad4809e9b18d2d01)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:09.276758 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:09.392671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:09.401558 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae290ceecd3e44b1ad4809e9b18d2d01', 'request_id': 'req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.407678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae290ceecd3e44b1ad4809e9b18d2d01', 'request_id': 'req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.414067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae290ceecd3e44b1ad4809e9b18d2d01', 'request_id': 'req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.511294 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:09.511757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:09.511863 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:09.531380 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] MFA Rules not processed for user `ea4ccbb465094f079dcc4ca3bcac59cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:09.532578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:09.549009 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:09.560910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] The token cache is being invalidate because project ae290ceecd3e44b1ad4809e9b18d2d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:09.561208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.561906 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.563443 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1370/2766] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:09 2022] DELETE /identity/v3/projects/ae290ceecd3e44b1ad4809e9b18d2d01 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:09.567902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:09.569320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:09.588303 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1397/2767] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:47:09.633683 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe1d1-bd20-4220-a6f6-939a2a21663a admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:09.638531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:09.639011 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:09.639299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:09.639577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] PATH_INFO: `/v3/users/a899d5319e904697903668fe203ef50c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:09.640784 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a899d5319e904697903668fe203ef50c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:09.642712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:09.879304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a899d5319e904697903668fe203ef50c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.880609 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a899d5319e904697903668fe203ef50c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.949433 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a899d5319e904697903668fe203ef50c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.952353 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a899d5319e904697903668fe203ef50c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.959887 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a899d5319e904697903668fe203ef50c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:09.982224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1371/2768] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:09 2022] DELETE /identity/v3/users/a899d5319e904697903668fe203ef50c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:47:10.209915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:10.211313 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:10.266443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745cb15-0e0c-42ba-983b-25cd35fef93e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:10.321376 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:10.322000 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:10.322300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:10.322551 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:10.324508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:10.390458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:10.390458 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:10.399179 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1398/2769] 10.222.0.144 () {60 vars in 1401 bytes} [Tue Jul 26 16:47:10 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:10.639066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:10.648517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:10.690677 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:10.712873 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7153e-9277-46ea-b4d2-d6c15d5c4e27 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:10.717734 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:10.718375 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:10.718761 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:10.719038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] PATH_INFO: `/v3/projects/db81b2f63e7b485f8161313fa6b06e4f` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:10.720178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db81b2f63e7b485f8161313fa6b06e4f)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:10.722161 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:10.952787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db81b2f63e7b485f8161313fa6b06e4f', 'request_id': 'req-99c952af-c471-41ea-af77-cc9f4b4041e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:10.959633 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db81b2f63e7b485f8161313fa6b06e4f', 'request_id': 'req-99c952af-c471-41ea-af77-cc9f4b4041e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:10.965222 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db81b2f63e7b485f8161313fa6b06e4f', 'request_id': 'req-99c952af-c471-41ea-af77-cc9f4b4041e4'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.051599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] The token cache is being invalidate because project db81b2f63e7b485f8161313fa6b06e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:11.051891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.055984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.055984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1372/2770] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:10 2022] DELETE /identity/v3/projects/db81b2f63e7b485f8161313fa6b06e4f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:11.058195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:11.069743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f080-e0cf-4370-887d-411836b1de7c service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:11.102872 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:11.126942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:11.127581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:11.127856 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:11.128127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] PATH_INFO: `/v3/users/331b058bd97f4a38bb6f13a30efca185` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:11.129307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] RBAC: Authorizing `identity:delete_user(user_id=331b058bd97f4a38bb6f13a30efca185)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:11.131890 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:11.349552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331b058bd97f4a38bb6f13a30efca185'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.350282 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331b058bd97f4a38bb6f13a30efca185'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.357288 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331b058bd97f4a38bb6f13a30efca185'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.357917 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331b058bd97f4a38bb6f13a30efca185'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.370103 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8324f1e7-caeb-45f5-9141-94d9183a283c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331b058bd97f4a38bb6f13a30efca185'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:11.449191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1399/2771] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:11 2022] DELETE /identity/v3/users/331b058bd97f4a38bb6f13a30efca185 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:47:11.850965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:11.852403 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:11.873499 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c952af-c471-41ea-af77-cc9f4b4041e4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:11.929913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:11.930559 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:11.930841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:11.931115 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] PATH_INFO: `/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:11.932333 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f72f058b9d4ef0897e28bf0b0774d1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:11.934318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.053224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f72f058b9d4ef0897e28bf0b0774d1', 'request_id': 'req-fc836fbf-5827-43a2-9551-a31105473298'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:12.059272 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f72f058b9d4ef0897e28bf0b0774d1', 'request_id': 'req-fc836fbf-5827-43a2-9551-a31105473298'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:12.064663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f72f058b9d4ef0897e28bf0b0774d1', 'request_id': 'req-fc836fbf-5827-43a2-9551-a31105473298'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:12.136696 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] The token cache is being invalidate because project e2f72f058b9d4ef0897e28bf0b0774d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:12.136982 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:12.137595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-fc836fbf-5827-43a2-9551-a31105473298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:12.139087 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1373/2772] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:11 2022] DELETE /identity/v3/projects/e2f72f058b9d4ef0897e28bf0b0774d1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:12.145165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.146197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.146197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.166138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:12.167341 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:12.226324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1400/2773] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:12.234669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.234964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.235244 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.243949 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:12.255676 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:12.256913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:12.265182 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1374/2774] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:12.269124 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.279695 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec546-635d-4c17-9a5a-b17f50f0fb1d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.284301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.285199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.285478 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.285878 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.287141 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:12.287443 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.289069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.304985 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1401/2775] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:47:12.306116 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.315445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619063a3-d85d-4f4c-8851-ea5a85459d0f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.321037 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.321740 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.322090 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.322451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.323197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.325114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.419657 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1375/2776] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/projects => generated 390 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:47:12.423842 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.433724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab2044-3c14-418f-ad92-14fc22d37831 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.474941 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:12.489428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.490062 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.490340 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.490612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.491391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.493358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.569768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1402/2777] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:47:12.574045 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.585176 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8239a-d724-4795-b9dc-1cb0644a20c4 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.589808 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.590478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.590738 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.591014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.591739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.593360 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.603290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1376/2778] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:12 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:47:12.607263 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.617343 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dede7-5c30-44d6-81f9-43869f19be3b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.622300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.622900 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.623188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.623463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.624165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.625765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d03cb-4a7a-41ab-9e0e-86196889897e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.638461 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1403/2779] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:12 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:12.642514 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.653230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002fb90-35b7-4f8b-b7f6-d3d873976b60 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.658061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.658713 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.658995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.659273 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] PATH_INFO: `/v3/projects/8de62e95e3ef45af81be02a834462d80/users/2485eeb38880439289ccbad17a562f74/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.674317 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de62e95e3ef45af81be02a834462d80, user_id=2485eeb38880439289ccbad17a562f74, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.677014 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:12.751917 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1377/2780] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:12 2022] PUT /identity/v3/projects/8de62e95e3ef45af81be02a834462d80/users/2485eeb38880439289ccbad17a562f74/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:12.763438 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.763728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.764030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.791635 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] MFA Rules not processed for user `2485eeb38880439289ccbad17a562f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:12.792870 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:12.874595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1404/2781] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:12.879203 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:12.940764 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5422e4-f259-4cfe-b3e5-dbae71176abb admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:12.946163 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:12.946773 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:12.947058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:12.947336 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:12.947994 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:12.949925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.053827 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1378/2782] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:12 2022] POST /identity/v3/projects => generated 386 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:13.057855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.069204 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a59ae-3789-4a10-b42a-3189ccf634c1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.123378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.123978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.124258 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.124562 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.125278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.127123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.217152 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1405/2783] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:13 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:13.221750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.233229 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf75834-abac-47b9-b9ec-c96916a1b3c2 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.237900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.238508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.238786 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.239105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.239798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.241417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.251992 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1379/2784] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:13 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:13.256828 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.266797 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f2c8-5e12-44c4-a8c8-754de8f9937a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.271967 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.272655 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.272936 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.273206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] PATH_INFO: `/v3/projects/771e771ef7f94a88a7df8ea651eafeea/users/13580bbce2a040c68f02efb28c119b7c/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.287650 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771e771ef7f94a88a7df8ea651eafeea, user_id=13580bbce2a040c68f02efb28c119b7c, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.290315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.374577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1406/2785] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:13 2022] PUT /identity/v3/projects/771e771ef7f94a88a7df8ea651eafeea/users/13580bbce2a040c68f02efb28c119b7c/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:13.378551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.389551 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433984-e5f6-4145-bee5-4c7bf711801f admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.447432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.448053 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.448337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.448655 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.449351 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.450937 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c3a2-3130-4193-879a-71cfc743cc7d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.461602 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1380/2786] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:13 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:13.465673 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.475926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7ee64-4865-400e-826e-9d6c97aec370 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.480587 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.481262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.481539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.481807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] PATH_INFO: `/v3/projects/771e771ef7f94a88a7df8ea651eafeea/users/13580bbce2a040c68f02efb28c119b7c/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.483732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771e771ef7f94a88a7df8ea651eafeea, user_id=13580bbce2a040c68f02efb28c119b7c, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.486381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.496379 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1407/2787] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:13 2022] PUT /identity/v3/projects/771e771ef7f94a88a7df8ea651eafeea/users/13580bbce2a040c68f02efb28c119b7c/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:13.508465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.508842 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.509153 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.533456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:13.542085 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] MFA Rules not processed for user `13580bbce2a040c68f02efb28c119b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:13.543366 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:13.610783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1381/2788] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:13.622508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.624190 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:13.630827 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:13.681482 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d5c39-d222-48a5-9513-b2dcaca5b881 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.685267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.685887 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.686175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.686453 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.695902 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.704153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d01d-a82e-4a21-a495-2c24f57f73a0 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:13.706962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1408/2789] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:13.851435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:13.912819 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c8f4a-264b-4187-9428-da2ebfa3460e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:13.917193 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:13.917830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:13.918110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:13.922549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] PATH_INFO: `/v3/users/2485eeb38880439289ccbad17a562f74` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:13.922549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2485eeb38880439289ccbad17a562f74)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:13.922549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:14.090527 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2485eeb38880439289ccbad17a562f74'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:14.091192 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2485eeb38880439289ccbad17a562f74'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:14.171891 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2485eeb38880439289ccbad17a562f74'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:14.172465 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2485eeb38880439289ccbad17a562f74'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:14.180227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2485eeb38880439289ccbad17a562f74'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:14.197418 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1382/2790] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:13 2022] DELETE /identity/v3/users/2485eeb38880439289ccbad17a562f74 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:14.201921 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:14.202234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:14.202505 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:14.220723 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:14.221958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:14.280307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1409/2791] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:14.290723 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:14.292132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:14.310804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:14.348975 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7093c3-fc9a-4492-b240-612c76274eb1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:14.352545 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:14.358447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:14.358447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:14.358447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:14.362404 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:14.363807 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:14.373575 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1383/2792] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:14.825480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:14.827937 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:14.839450 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231833d-f8f5-4d44-a66b-84d9bd24f1fd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:14.844399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:14.845049 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:14.845331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:14.845613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] PATH_INFO: `/v3/projects/8de62e95e3ef45af81be02a834462d80` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:14.846815 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de62e95e3ef45af81be02a834462d80)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:14.848762 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:14.991958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de62e95e3ef45af81be02a834462d80', 'request_id': 'req-c6723777-4aa7-404b-902d-54bc0ba52fe6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.000347 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de62e95e3ef45af81be02a834462d80', 'request_id': 'req-c6723777-4aa7-404b-902d-54bc0ba52fe6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.088674 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de62e95e3ef45af81be02a834462d80', 'request_id': 'req-c6723777-4aa7-404b-902d-54bc0ba52fe6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.099709 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] The token cache is being invalidate because project 8de62e95e3ef45af81be02a834462d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:15.099994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.100649 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.102183 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1410/2793] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:14 2022] DELETE /identity/v3/projects/8de62e95e3ef45af81be02a834462d80 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:15.106131 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:15.116288 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cda45-daf1-4f6a-a488-0dc558747712 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:15.178541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:15.179194 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:15.179467 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:15.179741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] PATH_INFO: `/v3/users/13580bbce2a040c68f02efb28c119b7c` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:15.181146 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13580bbce2a040c68f02efb28c119b7c)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:15.183177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:15.430651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13580bbce2a040c68f02efb28c119b7c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.431306 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13580bbce2a040c68f02efb28c119b7c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.506230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13580bbce2a040c68f02efb28c119b7c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.506871 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13580bbce2a040c68f02efb28c119b7c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.514257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-75df0701-193d-4d71-8725-426d88fc129b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13580bbce2a040c68f02efb28c119b7c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:15.532067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1384/2794] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:15 2022] DELETE /identity/v3/users/13580bbce2a040c68f02efb28c119b7c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:16.175623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:16.177745 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:16.195819 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6723777-4aa7-404b-902d-54bc0ba52fe6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:16.231561 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:16.266581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:16.267253 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.267560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.267830 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] PATH_INFO: `/v3/projects/771e771ef7f94a88a7df8ea651eafeea` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.269007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771e771ef7f94a88a7df8ea651eafeea)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:16.270954 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:16.415088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771e771ef7f94a88a7df8ea651eafeea', 'request_id': 'req-0e20a81f-faed-4cea-b379-c308813c0926'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:16.421531 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771e771ef7f94a88a7df8ea651eafeea', 'request_id': 'req-0e20a81f-faed-4cea-b379-c308813c0926'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:16.500679 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771e771ef7f94a88a7df8ea651eafeea', 'request_id': 'req-0e20a81f-faed-4cea-b379-c308813c0926'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:16.511306 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] The token cache is being invalidate because project 771e771ef7f94a88a7df8ea651eafeea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:16.511593 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:16.512237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0e20a81f-faed-4cea-b379-c308813c0926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:16.513785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1411/2795] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:16 2022] DELETE /identity/v3/projects/771e771ef7f94a88a7df8ea651eafeea => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:16.518793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.519334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.519334 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.539592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:16.540845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:16.577754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:16.603714 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1385/2796] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:16.611570 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.613113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.613113 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.633240 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:16.634469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:16.642466 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1412/2797] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:16.647508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:16.657939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290dd59-a26a-4a79-bf6d-0e21bffeaa89 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:16.663445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:16.664065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.664346 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.664643 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.665350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:16.665653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:16.667240 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:16.682276 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1386/2798] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:47:16.683526 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:16.693097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a30688-8cc5-488b-8fbb-410ade8fbd87 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:16.698171 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:16.698859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.699216 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.699566 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.700480 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:16.702464 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:16.849720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1413/2799] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:16 2022] POST /identity/v3/projects => generated 368 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:16.854213 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:16.870647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085834-9f2d-4cac-adeb-483c30b9b3d8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:16.943899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:16.944661 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:16.945023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:16.945379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:16.946065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:16.947984 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.068712 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1387/2800] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:16 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:17.072659 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.084121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fb1a1-45b3-4748-9485-584b2cf442f1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.089059 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.089682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.090066 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.090272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.090960 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.092612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.103167 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1414/2801] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:17 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:17.107260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.120243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbe7a0-bf16-41b7-90b7-514ed62a540d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.126669 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.127365 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.127989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.127989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.128813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.134498 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c66f8-5f07-4e2a-b464-419e5c3ff24f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.147104 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1388/2802] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:17 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:17.148300 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.158850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416f931-20be-429a-b691-e5e24b2f0f1e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.163756 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.164922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.165207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.165469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] PATH_INFO: `/v3/projects/61fc03e96db9417292c47192ab754306/users/3eac0268759f4760a07df95275c9610d/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.185668 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61fc03e96db9417292c47192ab754306, user_id=3eac0268759f4760a07df95275c9610d, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.188342 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.301057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1415/2803] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:17 2022] PUT /identity/v3/projects/61fc03e96db9417292c47192ab754306/users/3eac0268759f4760a07df95275c9610d/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:17.312768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.313054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.313332 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.342078 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] MFA Rules not processed for user `3eac0268759f4760a07df95275c9610d`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:17.343292 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:17.407647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1389/2804] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:17.412615 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.470733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:17.474644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55a7d4-5576-4632-859d-aa9acbd57c5c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.479367 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.479983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.480267 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.480644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.481358 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.483297 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.595007 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1416/2805] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:17 2022] POST /identity/v3/projects => generated 368 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:17.598794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.612556 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c4893-15c8-4d4f-9318-8c136cddd560 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.652831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:17.687245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.688088 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.688474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.688833 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.689813 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.692811 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.804815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1390/2806] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:17 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:17.808539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.818915 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f207-4c54-4ab7-8a78-4aa9281921f3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.823702 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.824369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.824760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.825135 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.825884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.827682 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.839638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1417/2807] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:17 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:17.843995 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:17.861914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64853b9e-5aaa-431a-a566-f59a766954f1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:17.861914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:17.861914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:17.861914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:17.862328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] PATH_INFO: `/v3/projects/34c1799bd97740588e827b81c0420f60/users/c741ad32a21f4a9f82c7352752099ac1/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:17.876177 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c1799bd97740588e827b81c0420f60, user_id=c741ad32a21f4a9f82c7352752099ac1, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:17.878856 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:17.985531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1391/2808] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:17 2022] PUT /identity/v3/projects/34c1799bd97740588e827b81c0420f60/users/c741ad32a21f4a9f82c7352752099ac1/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:47:17.990457 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:18.003312 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d84dae-2c01-4851-a28f-27b10e4ea655 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:18.071517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:18.072149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:18.072501 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:18.072776 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:18.073467 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:18.075175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a583a4-784a-4173-9962-75c9052a3fbe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:18.086813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1418/2809] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:17 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:18.091459 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:18.102603 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe97d1-35b9-4d26-813f-c190edfc5dab admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:18.107438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:18.108118 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:18.108424 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:18.114588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] PATH_INFO: `/v3/projects/34c1799bd97740588e827b81c0420f60/users/c741ad32a21f4a9f82c7352752099ac1/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:18.114588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c1799bd97740588e827b81c0420f60, user_id=c741ad32a21f4a9f82c7352752099ac1, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:18.114588 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:18.125280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1392/2810] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:18 2022] PUT /identity/v3/projects/34c1799bd97740588e827b81c0420f60/users/c741ad32a21f4a9f82c7352752099ac1/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:18.135728 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:18.136091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:18.137459 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:18.168109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] MFA Rules not processed for user `c741ad32a21f4a9f82c7352752099ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:18.172428 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:18.245499 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1419/2811] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:18.253022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:18.254767 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:18.322123 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa12ef4-7729-4390-abaf-cd9bb0d67ca3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:18.325928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:18.326549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:18.326830 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:18.327133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:18.337004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:18.338425 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:18.347836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1393/2812] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:18.590884 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:18.601390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0232b69-c4d0-4abf-896c-f783feca9b7b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:18.604909 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:18.605514 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:18.605792 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:18.606061 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:18.658416 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:18.667733 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:18.669206 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ce16e-db56-4b5e-8ddb-94064d2e1ac4 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:18.678158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1420/2813] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:28.343645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:29.000287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:30.175909 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:30.235482 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd847d6-04d3-4dab-b3f5-2555b21b3b44 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:30.239948 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:30.240598 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:30.240877 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:30.241148 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] PATH_INFO: `/v3/users/3eac0268759f4760a07df95275c9610d` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:30.242293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eac0268759f4760a07df95275c9610d)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:30.244358 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:30.476962 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eac0268759f4760a07df95275c9610d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:30.477595 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eac0268759f4760a07df95275c9610d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:30.572733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eac0268759f4760a07df95275c9610d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:30.573362 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eac0268759f4760a07df95275c9610d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:30.581033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eac0268759f4760a07df95275c9610d'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:30.599739 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1394/2814] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:30 2022] DELETE /identity/v3/users/3eac0268759f4760a07df95275c9610d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:30.604813 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:30.605284 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:30.605386 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:30.623868 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:30.625104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:30.683133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1421/2815] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:30.692868 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:30.694285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:30.751513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7a3a7-4414-4707-99ba-52c20564d485 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:30.755096 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:30.755704 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:30.755985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:30.756260 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:30.766012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:30.767432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:30.777107 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1395/2816] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:31.359769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:31.361293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:31.377030 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67a68-75a4-420c-a1b0-7dc99e6f0a27 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:31.382754 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:31.383907 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:31.384197 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:31.384494 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] PATH_INFO: `/v3/projects/61fc03e96db9417292c47192ab754306` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:31.385711 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=61fc03e96db9417292c47192ab754306)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:31.388720 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:31.538405 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61fc03e96db9417292c47192ab754306', 'request_id': 'req-82d4dba9-db89-4fba-a774-7c1192f6a8ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.544737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61fc03e96db9417292c47192ab754306', 'request_id': 'req-82d4dba9-db89-4fba-a774-7c1192f6a8ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.625918 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61fc03e96db9417292c47192ab754306', 'request_id': 'req-82d4dba9-db89-4fba-a774-7c1192f6a8ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.637755 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] The token cache is being invalidate because project 61fc03e96db9417292c47192ab754306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:31.638047 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.638743 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.640369 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1422/2817] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:31 2022] DELETE /identity/v3/projects/61fc03e96db9417292c47192ab754306 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:31.644486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:31.654020 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97effe-adae-4ed6-a86f-a76537bbe3fa service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:31.672413 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:31.714680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:31.715324 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:31.715604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:31.715875 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] PATH_INFO: `/v3/users/c741ad32a21f4a9f82c7352752099ac1` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:31.717067 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c741ad32a21f4a9f82c7352752099ac1)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:31.719031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:31.945466 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c741ad32a21f4a9f82c7352752099ac1'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:31.946212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c741ad32a21f4a9f82c7352752099ac1'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:32.097289 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c741ad32a21f4a9f82c7352752099ac1'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:32.097981 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c741ad32a21f4a9f82c7352752099ac1'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:32.106031 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c741ad32a21f4a9f82c7352752099ac1'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:32.126206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1396/2818] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:31 2022] DELETE /identity/v3/users/c741ad32a21f4a9f82c7352752099ac1 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:33.176010 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:33.179520 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:33.191084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4dba9-db89-4fba-a774-7c1192f6a8ee admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:33.250812 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:33.251541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.251823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.252160 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] PATH_INFO: `/v3/projects/34c1799bd97740588e827b81c0420f60` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.253391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c1799bd97740588e827b81c0420f60)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:33.255319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:33.379109 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:33.395657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c1799bd97740588e827b81c0420f60', 'request_id': 'req-30ec4bcb-0258-45fb-b89a-333b380fec9d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:33.401790 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c1799bd97740588e827b81c0420f60', 'request_id': 'req-30ec4bcb-0258-45fb-b89a-333b380fec9d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:33.504563 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:33.506694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:33.520618 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c1799bd97740588e827b81c0420f60', 'request_id': 'req-30ec4bcb-0258-45fb-b89a-333b380fec9d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:33.532610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] The token cache is being invalidate because project 34c1799bd97740588e827b81c0420f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:33.532898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:33.533568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-30ec4bcb-0258-45fb-b89a-333b380fec9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:33.538846 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1423/2819] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:33 2022] DELETE /identity/v3/projects/34c1799bd97740588e827b81c0420f60 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:33.540850 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b69d4-fda1-4a65-b72f-08184ed97f45 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.541138 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b69d4-fda1-4a65-b72f-08184ed97f45 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.541411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b69d4-fda1-4a65-b72f-08184ed97f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.562624 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1c1b69d4-fda1-4a65-b72f-08184ed97f45 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:33.564248 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1c1b69d4-fda1-4a65-b72f-08184ed97f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:33.571501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db290-1ec8-4e7a-b685-a6c6fd8871b7 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:33.627178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:33.627939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.628741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.628741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.630541 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:33.653127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:33.666026 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1424/2820] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:33.676644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.676939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.677213 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.703012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:33.703012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:33.708970 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1425/2821] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:33.713274 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:33.718449 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:33.720026 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:33.722571 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ded95-dd86-42de-9a28-3f54ad9d7f85 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:33.728737 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:33.729751 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.730039 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.730319 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.731272 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:33.731549 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:33.732699 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1397/2822] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:47:33.733291 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:33.752165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1426/2823] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:47:33.753793 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:33.764750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e9c2a-d7a5-4871-b2c4-4f3ed81263cc service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:33.769528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:33.770380 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.770745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.771101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.771879 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:33.773744 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:33.882214 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1398/2824] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:33 2022] POST /identity/v3/projects => generated 386 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:33.886401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:33.896454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da7b8-033d-4602-89ad-cfe495e3d5e6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:33.953809 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:33.954345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:33.954626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:33.954901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:33.955612 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:33.957568 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:34.092214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1427/2825] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:33 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:34.096370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:34.107102 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a142a2-efc0-406b-b31b-1bcb1b76cd94 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:34.111631 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:34.112284 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.112599 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.112881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.113583 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:34.115164 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:34.125900 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1399/2826] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:34 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:34.129861 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:34.140199 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d44da-be94-4d96-9c07-cdadad1e076b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:34.144757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:34.145350 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.145629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.145901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.146578 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:34.148229 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f3277-3852-4cb8-a55d-963ddd10b9ce admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:34.159031 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1428/2827] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:34 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:34.163208 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:34.173099 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63c06-7665-447e-9528-cc551ca8d4a3 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:34.177667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:34.178313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.178592 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.178866 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] PATH_INFO: `/v3/projects/495454a9410c42069ffb7900ccd4b852/users/fb95ba86d35141b4901ab6af7b1e2dad/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.193907 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] RBAC: Authorizing `identity:create_grant(project_id=495454a9410c42069ffb7900ccd4b852, user_id=fb95ba86d35141b4901ab6af7b1e2dad, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:34.197186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:34.306670 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1400/2828] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:34 2022] PUT /identity/v3/projects/495454a9410c42069ffb7900ccd4b852/users/fb95ba86d35141b4901ab6af7b1e2dad/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:34.318088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.318598 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.318713 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.346243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] MFA Rules not processed for user `fb95ba86d35141b4901ab6af7b1e2dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:34.347437 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:34.409774 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1429/2829] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:34.420965 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:34.422650 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:34.478629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419626-8f92-437a-93ef-126a93cc747b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:34.482378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:34.483022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.483309 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.483582 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.493486 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:34.495562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:34.504530 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1401/2830] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:34.851807 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:34.925510 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:34.926913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:34.990639 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039b164-24b4-4b3a-afb5-3724a6af296d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:34.994942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:34.995617 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:34.995898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:34.996242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:34.997939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:35.067127 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:35.068580 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2c34-c4c6-407d-bd2d-65df9fd2ab6f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:35.079610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1430/2831] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:36.012729 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:36.014055 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:36.061310 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:36.080110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6b6d8-056c-4fe9-ab02-6b41c7bc3ad3 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:36.084694 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:36.085325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:36.085604 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:36.085985 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] PATH_INFO: `/v3/users/1c197746f76549e6923e961aceb9b1c3` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:36.087185 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c197746f76549e6923e961aceb9b1c3)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:36.089108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:36.408111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c197746f76549e6923e961aceb9b1c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:36.408778 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c197746f76549e6923e961aceb9b1c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:36.547718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c197746f76549e6923e961aceb9b1c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:36.547718 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c197746f76549e6923e961aceb9b1c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:36.554578 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c197746f76549e6923e961aceb9b1c3'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:36.572535 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1402/2832] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:36 2022] DELETE /identity/v3/users/1c197746f76549e6923e961aceb9b1c3 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:36.577173 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:36.577645 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:36.577769 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:36.596608 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:36.597823 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:36.666764 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1431/2833] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:36.677700 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:36.679113 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:36.737724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee258b3c-0459-449e-b964-e70926d643ec admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:36.741492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:36.742105 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:36.742409 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:36.742680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:36.752009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:36.753448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d45a9b-149e-44f0-8964-8ef1d7343e00 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:36.763350 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1403/2834] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:36.986942 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:37.498783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:37.499161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:37.514209 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56a716-8d44-455a-9ebe-0073ca6b56cd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:37.521524 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:37.522194 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:37.533397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:37.533397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] PATH_INFO: `/v3/projects/5a64710ed3bf46fdae206b853de3ee2d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:37.533397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a64710ed3bf46fdae206b853de3ee2d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:37.533397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:37.660611 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a64710ed3bf46fdae206b853de3ee2d', 'request_id': 'req-1bfab1b3-ccd7-4e40-98cb-1855344e6738'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:37.666928 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a64710ed3bf46fdae206b853de3ee2d', 'request_id': 'req-1bfab1b3-ccd7-4e40-98cb-1855344e6738'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:37.768981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a64710ed3bf46fdae206b853de3ee2d', 'request_id': 'req-1bfab1b3-ccd7-4e40-98cb-1855344e6738'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:37.779906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] The token cache is being invalidate because project 5a64710ed3bf46fdae206b853de3ee2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:37.780192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:37.780855 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1bfab1b3-ccd7-4e40-98cb-1855344e6738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:37.782401 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1432/2835] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:37 2022] DELETE /identity/v3/projects/5a64710ed3bf46fdae206b853de3ee2d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:37.787528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:37.788023 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:37.788128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:37.808898 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:37.810690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:37.870423 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1404/2836] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:37.878916 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:37.879208 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:37.879490 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:37.900924 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:37.902149 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:37.913873 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1433/2837] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:37.915110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:37.926143 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2008-1f47-42e4-b176-4afafe7afabb None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:37.932950 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:37.933945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:37.934414 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:37.934890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:37.936722 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:37.937168 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:37.938968 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:37.950590 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1405/2838] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:47:37.954997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:37.964181 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dc51d-a3ea-440d-b199-3f5b65c3d5ea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:37.968825 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:37.969448 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:37.969725 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:37.969993 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:37.970644 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:37.972681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.079642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1434/2839] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:37 2022] POST /identity/v3/projects => generated 372 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:38.083647 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.093967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c3609-2360-4f84-ad8f-e9443d7a067c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.158101 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.158721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.159005 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.159287 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.160219 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.162379 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.218452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:38.287038 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1406/2840] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:38 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:38.291203 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.301997 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0483-b1ce-4486-903c-46f9dd58eb88 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.306486 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.307090 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.307393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.308454 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.309301 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.311123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.322081 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1435/2841] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:38 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:38.326133 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.336257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffae1c-8919-437c-b303-4651659836bf admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.341065 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.342206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.342513 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.342783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.343518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.345128 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026d60-2bf7-4157-942e-5b1c5d65760b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.360091 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1407/2842] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:38 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:38.361161 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.371243 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bcfc-1a8d-4e05-b237-6c67574b7998 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.375964 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.376729 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.377012 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.377290 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] PATH_INFO: `/v3/projects/af6b766b3c194614add7413dba56151f/users/ca1e11a1c42f45ffa74b430448105b8a/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.392022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6b766b3c194614add7413dba56151f, user_id=ca1e11a1c42f45ffa74b430448105b8a, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.400820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.523201 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1436/2843] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:38 2022] PUT /identity/v3/projects/af6b766b3c194614add7413dba56151f/users/ca1e11a1c42f45ffa74b430448105b8a/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:38.535939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.536230 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.536725 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.564695 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] MFA Rules not processed for user `ca1e11a1c42f45ffa74b430448105b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:38.565925 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:38.630363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1408/2844] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:38.642765 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.644738 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:38.707479 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076f4ad-af7c-4cb4-ba2a-bb1c16417f1d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.711169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.711854 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.712202 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.712589 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.725414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.725414 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95df44d-9775-46b8-b903-62498260f035 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:38.733613 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1437/2845] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:38.758474 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:38.820110 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361bf2-9491-4ae8-9a6a-23f2ca6cfc33 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:38.825186 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:38.826947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:38.826947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:38.826947 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] PATH_INFO: `/v3/users/ca1e11a1c42f45ffa74b430448105b8a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:38.828233 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1e11a1c42f45ffa74b430448105b8a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:38.830205 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:39.073945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1e11a1c42f45ffa74b430448105b8a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:39.074611 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1e11a1c42f45ffa74b430448105b8a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:39.145863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1e11a1c42f45ffa74b430448105b8a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:39.146495 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1e11a1c42f45ffa74b430448105b8a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:39.153802 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1e11a1c42f45ffa74b430448105b8a'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:39.172293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1409/2846] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:38 2022] DELETE /identity/v3/users/ca1e11a1c42f45ffa74b430448105b8a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:39.176958 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:39.177249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:39.177554 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:39.191577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:39.197820 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:39.199038 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:39.261451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:39.261687 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1438/2847] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:39.270042 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:39.271641 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:39.333410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dca00-481d-4d11-8136-ebbf9b5b68b0 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:39.337239 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:39.341902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:39.341902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:39.341902 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:39.348407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:39.350066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d84937-2ca5-4e5a-b08f-6b21fa97369c service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:39.361518 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1410/2848] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:39.811539 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:39.813211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:39.825560 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e809-c9e2-46a5-b75e-e975c590463d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:39.838411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:39.839469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:39.839469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:39.839642 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] PATH_INFO: `/v3/projects/af6b766b3c194614add7413dba56151f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:39.841586 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6b766b3c194614add7413dba56151f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:39.843798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.014869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6b766b3c194614add7413dba56151f', 'request_id': 'req-4fdb730c-4326-41f0-809b-6d9648d280a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:40.021159 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6b766b3c194614add7413dba56151f', 'request_id': 'req-4fdb730c-4326-41f0-809b-6d9648d280a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:40.149165 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6b766b3c194614add7413dba56151f', 'request_id': 'req-4fdb730c-4326-41f0-809b-6d9648d280a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:40.160757 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] The token cache is being invalidate because project af6b766b3c194614add7413dba56151f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:40.161046 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:40.161670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4fdb730c-4326-41f0-809b-6d9648d280a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:40.163175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1439/2849] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:39 2022] DELETE /identity/v3/projects/af6b766b3c194614add7413dba56151f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:40.169290 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.169801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.169914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.189132 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:40.190328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:40.250501 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1411/2850] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:40.258214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.258497 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.258767 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.279657 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:40.280932 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:40.289006 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1440/2851] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:40.293122 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.302967 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474717-e254-4e51-940a-d741d3e02b9e None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.307416 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.308015 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.308293 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.308610 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.309300 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80369) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:40.309600 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.311241 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.322417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1412/2852] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:47:40.327397 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.336739 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a84cf-8e65-4f50-915b-9044bff69e77 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.341969 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.342569 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.342849 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.343120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.343777 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.345785 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.499278 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1441/2853] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:40 2022] POST /identity/v3/projects => generated 386 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:40.503402 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.513364 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28df4e-16ee-4455-bff6-76834e430255 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.521589 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:40.570580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.571188 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.571478 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.571749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.572429 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.574319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.726363 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1413/2854] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:40 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:40.730254 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.740509 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea1726-c4f7-44e8-b00d-013797f6f50e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.745084 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.745684 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.745965 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.746242 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.746925 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.749042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.759036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1442/2855] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:40 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:40.762872 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.773548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30c7e-25a0-45c8-a477-0422d36b4ed1 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.778299 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.778894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.779172 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.779447 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.780126 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.781721 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccf3f2-c800-4da5-a5d3-4dddf20311a3 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.792791 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1414/2856] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:40 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:40.797261 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:40.806898 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278fa4d-0223-4a06-b83f-ccad6523ac94 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:40.811994 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:40.812681 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.812959 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.813233 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] PATH_INFO: `/v3/projects/417fd6ab3a0c4537a10272ee14f1cadb/users/e57b5c97a19a42c28156f1fc82384fc1/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.827316 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=417fd6ab3a0c4537a10272ee14f1cadb, user_id=e57b5c97a19a42c28156f1fc82384fc1, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:40.829975 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:40.972676 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1443/2857] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:40 2022] PUT /identity/v3/projects/417fd6ab3a0c4537a10272ee14f1cadb/users/e57b5c97a19a42c28156f1fc82384fc1/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:40.986770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:40.986770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:40.986770 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:40.997216 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:40.998503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:41.055639 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1415/2858] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:41.065150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:41.066859 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:41.099116 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:41.124440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e5468-61a0-44d0-a193-5cf87a75f41a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:41.127978 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:41.128614 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:41.128893 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:41.129162 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:41.138407 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:41.139821 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cce87-3363-46cf-b8b3-eb44dd31a922 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:41.152244 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1444/2859] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:47.658457 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:47.659928 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:47.674930 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567db49-09f4-4dc9-a11a-2baf4b8f002f None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:47.679955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:47.680645 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:47.680927 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:47.681201 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] PATH_INFO: `/v3/users/fb95ba86d35141b4901ab6af7b1e2dad` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:47.682406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb95ba86d35141b4901ab6af7b1e2dad)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:47.684318 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:47.955685 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:47.956195 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:47.956321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:47.983914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] MFA Rules not processed for user `e57b5c97a19a42c28156f1fc82384fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:47.985399 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:48.027691 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb95ba86d35141b4901ab6af7b1e2dad'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:48.028667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb95ba86d35141b4901ab6af7b1e2dad'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:48.056022 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1445/2860] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:47:48.061391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.071783 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd012325-47bf-4938-b7c0-92db0c9b5307 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.076973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.077585 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.077867 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.078142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.078802 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.080716 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.123841 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb95ba86d35141b4901ab6af7b1e2dad'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:48.124577 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb95ba86d35141b4901ab6af7b1e2dad'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:48.132528 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-8947eeb4-fe98-4f76-9155-39ce9b5b670f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb95ba86d35141b4901ab6af7b1e2dad'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:48.150491 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1416/2861] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:47:47 2022] DELETE /identity/v3/users/fb95ba86d35141b4901ab6af7b1e2dad => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:48.154741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.157763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.157763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.162481 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:48.174387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:48.175572 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:48.202156 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1446/2862] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:48 2022] POST /identity/v3/projects => generated 386 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:48.206324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.207944 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:48.234071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1417/2863] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 16:47:48.244114 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.245490 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:48.272469 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83606b6-72ae-4c79-9da0-67d91f6ff01f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.277392 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.280168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.280168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.280168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.280168 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.281465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.303378 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ead71-ba38-44d1-af0f-179f93fe7a77 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.353475 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.354127 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.354406 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.354679 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.364407 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.365964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.375814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1418/2864] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:47:48.384191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1447/2865] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:48 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:48.387058 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.398492 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69e94-2067-44c6-916b-fae8371cd0bf service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.404136 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.404893 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.405178 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.405448 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.406149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.407733 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.418387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1419/2866] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:48 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:48.422411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.433355 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596352f-78a8-403d-9c1b-102f9d2d76d1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.438391 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.439071 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.439396 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.439675 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] PATH_INFO: `/v3/projects/c60bfae6ddba400dba3276c5c5e7832a/users/d6f1b172408d419eba93becb942005bc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.456169 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60bfae6ddba400dba3276c5c5e7832a, user_id=d6f1b172408d419eba93becb942005bc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.458869 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.578218 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1448/2867] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:48 2022] PUT /identity/v3/projects/c60bfae6ddba400dba3276c5c5e7832a/users/d6f1b172408d419eba93becb942005bc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:48.582325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.592953 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc40a3-cf23-428d-86ee-6f0817ac81af admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.644671 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:48.661141 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.661771 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.662054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.662330 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.663033 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.664709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.678079 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1420/2868] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:48 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:48.682933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.689249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:48.695805 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ca95e-bb89-4f5d-9205-a176761f48fc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:48.701528 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:48.708973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:48.708973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:48.708973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] PATH_INFO: `/v3/projects/c60bfae6ddba400dba3276c5c5e7832a/users/d6f1b172408d419eba93becb942005bc/roles/96e631f147a74de7b904e1ecc4b4d8f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:48.708973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60bfae6ddba400dba3276c5c5e7832a, user_id=d6f1b172408d419eba93becb942005bc, role_id=96e631f147a74de7b904e1ecc4b4d8f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:48.708973 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8bd98-0eb4-4964-aebd-69e93fd670af admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:48.722938 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1449/2869] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:48 2022] PUT /identity/v3/projects/c60bfae6ddba400dba3276c5c5e7832a/users/d6f1b172408d419eba93becb942005bc/roles/96e631f147a74de7b904e1ecc4b4d8f8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:48.991646 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:48.993730 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:49.019303 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165e51-7ddb-40a2-8dc6-84d67a43200d admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.092108 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.098280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.098280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.098280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] PATH_INFO: `/v3/projects/495454a9410c42069ffb7900ccd4b852` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.098280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] RBAC: Authorizing `identity:delete_project(project_id=495454a9410c42069ffb7900ccd4b852)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.098280 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.224768 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495454a9410c42069ffb7900ccd4b852', 'request_id': 'req-d691bc52-818a-4e24-8713-bc84f692f67b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:49.232451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495454a9410c42069ffb7900ccd4b852', 'request_id': 'req-d691bc52-818a-4e24-8713-bc84f692f67b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:49.239054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495454a9410c42069ffb7900ccd4b852', 'request_id': 'req-d691bc52-818a-4e24-8713-bc84f692f67b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:49.319328 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] The token cache is being invalidate because project 495454a9410c42069ffb7900ccd4b852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:47:49.319627 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:49.320243 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-d691bc52-818a-4e24-8713-bc84f692f67b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:47:49.321851 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1421/2870] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:47:48 2022] DELETE /identity/v3/projects/495454a9410c42069ffb7900ccd4b852 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:49.326726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.327186 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.327283 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.347424 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:49.348690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:49.433603 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1450/2871] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:49.441693 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.442004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.443093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.464899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:49.466659 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:49.479072 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1422/2872] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:49.484329 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.495142 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a566548-bc38-4f1e-b52d-9a9edf93320f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.500001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.500744 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.501104 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.501456 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.502256 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:47:49.502581 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.504331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.524737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.528956 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1451/2873] 10.222.0.144 () {58 vars in 1158 bytes} [Tue Jul 26 16:47:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:47:49.535782 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8440120-ddb2-4fdb-8404-c77dea492ab9 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.540663 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.541285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.541749 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.542030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.542698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.545384 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.655497 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1423/2874] 10.222.0.144 () {60 vars in 1158 bytes} [Tue Jul 26 16:47:49 2022] POST /identity/v3/projects => generated 378 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:47:49.656763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.670211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64314-4e54-4bf4-86b5-f66c405d2601 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.747840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.748629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.748981 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.749331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.750089 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.752077 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.852037 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1452/2875] 10.222.0.144 () {60 vars in 1149 bytes} [Tue Jul 26 16:47:49 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:47:49.856777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.867355 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c72bd-31de-4b39-abde-3b18976c2d21 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.873678 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.875689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.875689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.875689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.875689 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.877964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.895503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1424/2876] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:49 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:49.900933 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.917307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6faf1-3a5d-408c-969f-432e45f4cc63 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.922724 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.923495 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.923891 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.924304 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.925132 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:49.926857 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb2402-0eea-4662-b1ed-20944203bb65 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:49.944892 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1453/2877] 10.222.0.144 () {58 vars in 1127 bytes} [Tue Jul 26 16:47:49 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:47:49.949047 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:49.964030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319161a-2d09-4e32-a3de-47e0768606c8 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:49.970093 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:49.970989 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:49.971370 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:49.971843 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] PATH_INFO: `/v3/projects/6b3ed5a8c695456ca0edf53ad0d82bfe/users/b6f2263a37db4f72968f397b7e6ffcca/roles/5a1e89be85c547438247b00c04c09046` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:49.995787 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3ed5a8c695456ca0edf53ad0d82bfe, user_id=b6f2263a37db4f72968f397b7e6ffcca, role_id=5a1e89be85c547438247b00c04c09046)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:50.000257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:50.149658 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1425/2878] 10.222.0.144 () {60 vars in 1488 bytes} [Tue Jul 26 16:47:49 2022] PUT /identity/v3/projects/6b3ed5a8c695456ca0edf53ad0d82bfe/users/b6f2263a37db4f72968f397b7e6ffcca/roles/5a1e89be85c547438247b00c04c09046 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:47:50.155750 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6fdfd-7b2a-4e69-8f9c-04764e0a4d95 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:50.156080 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6fdfd-7b2a-4e69-8f9c-04764e0a4d95 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:50.156310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6fdfd-7b2a-4e69-8f9c-04764e0a4d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:50.183440 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-23f6fdfd-7b2a-4e69-8f9c-04764e0a4d95 None None] MFA Rules not processed for user `b6f2263a37db4f72968f397b7e6ffcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:50.184697 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-23f6fdfd-7b2a-4e69-8f9c-04764e0a4d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:50.229987 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:50.254050 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1454/2879] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:50.265456 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:50.267743 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:50.286437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:50.338863 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccd8f5-a169-44a3-926c-99c6c78c5e54 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:50.340532 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5361a7650f164ff8b11abfe6b3c4c382', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:50.341910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:50.342257 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:50.342750 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:50.353698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:50.355343 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:50.365763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1426/2880] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:55.143178 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248303f-6b6b-435b-965b-a0570d12b1d6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:55.143760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248303f-6b6b-435b-965b-a0570d12b1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:55.143760 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248303f-6b6b-435b-965b-a0570d12b1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:55.174365 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f248303f-6b6b-435b-965b-a0570d12b1d6 None None] MFA Rules not processed for user `d6f1b172408d419eba93becb942005bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:55.175507 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f248303f-6b6b-435b-965b-a0570d12b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:55.258067 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1455/2881] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:55.271934 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:55.273419 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:55.348022 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af775b-e24c-4298-962d-b4c6767c0c65 service cinder] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:55.351884 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6d652014d6446a0ae162120aac0ed94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:55.352570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:55.352870 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:55.353285 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:55.355142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:55.425698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:55.427095 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:55.440580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1427/2882] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:47:55.710187 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb2b-fb90-4ceb-8678-300f6fb2eca7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:55.710475 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb2b-fb90-4ceb-8678-300f6fb2eca7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:55.710752 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb2b-fb90-4ceb-8678-300f6fb2eca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:55.730950 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fb0ceb2b-fb90-4ceb-8678-300f6fb2eca7 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:47:55.732185 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fb0ceb2b-fb90-4ceb-8678-300f6fb2eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:55.798771 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1456/2883] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:47:55.927836 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:47:55.929337 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:47:55.999894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eef45-775e-40bb-b5d7-fcd3762f2848 service glance] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:47:56.004783 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:47:56.005417 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:47:56.005697 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:47:56.005970 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:47:56.014759 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:47:56.016335 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:47:56.021061 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=100 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:47:56.025160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1428/2884] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:48:05.788463 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=66 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:06.003794 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:06.004091 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:06.004370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:06.023734 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] MFA Rules not processed for user `e57b5c97a19a42c28156f1fc82384fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:48:06.024989 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:06.031988 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1457/2885] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:48:06.287910 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:06.289531 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:06.345903 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa38718-ebed-4717-b370-84d1c532f1f5 service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:06.349477 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:06.350081 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:06.350359 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:06.350629 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:06.352432 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:06.362227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:06.363667 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a7273-8f30-4e11-a322-89d1c91aa201 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:06.372565 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1429/2886] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:48:06.808796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:06.811001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:06.885191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e3b40-5541-48c2-a386-a2e2536cc84b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:06.890264 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:06.890914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:06.891191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:06.891465 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:06.893214 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:06.905070 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:06.906648 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:06.916953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1458/2887] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:48:10.381455 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] REQUEST_METHOD: `POST` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:10.382002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:10.382002 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:10.401913 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.auth.core [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80369) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:48:10.402304 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:10.410562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1430/2888] 10.222.0.144 () {58 vars in 1014 bytes} [Tue Jul 26 16:48:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:48:10.423262 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:10.434796 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573da9e0-b627-4a3f-b6ad-469a944a4a9f service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:10.439726 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:10.440323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:10.440629 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:10.440901 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:10.449192 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:10.450922 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7107789-3d1e-41e8-abf7-f31e9d67f10d service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:10.482896 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1459/2889] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:48:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:48:20.395576 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=29 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:20.470333 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=41 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:23.700681 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:23.761880 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe1bfd-9a4f-462d-85af-7033430d0ad0 None None] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:23.766305 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:23.766940 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:23.767227 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:23.767505 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] PATH_INFO: `/v3/users/b6f2263a37db4f72968f397b7e6ffcca` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:23.768815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f2263a37db4f72968f397b7e6ffcca)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:23.770741 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:23.971277 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f2263a37db4f72968f397b7e6ffcca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:23.971955 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f2263a37db4f72968f397b7e6ffcca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:24.062224 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f2263a37db4f72968f397b7e6ffcca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:24.063012 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f2263a37db4f72968f397b7e6ffcca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:24.072653 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f2263a37db4f72968f397b7e6ffcca'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:24.095314 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1431/2890] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:48:23 2022] DELETE /identity/v3/users/b6f2263a37db4f72968f397b7e6ffcca => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:48:24.100953 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:24.101500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:24.101500 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:24.121455 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] MFA Rules not processed for user `d88063924d80461f80d0ee08a35cdcfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:48:24.122670 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:24.189840 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1460/2891] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:48:24.216503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:24.218622 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:24.280890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645921-c785-4b6c-8e30-cdb4b0a55e3b admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:24.284584 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:24.285197 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:24.285746 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:24.285996 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:24.295801 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:24.297320 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:24.309345 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1432/2892] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:48:24.798036 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:24.799810 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:24.827911 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241596da-d03d-4287-9cb0-7cef5f9b7870 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:24.838207 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:24.839337 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:24.839824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:24.840508 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] PATH_INFO: `/v3/projects/6b3ed5a8c695456ca0edf53ad0d82bfe` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:24.842133 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3ed5a8c695456ca0edf53ad0d82bfe)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:24.844906 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:24.971172 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3ed5a8c695456ca0edf53ad0d82bfe', 'request_id': 'req-f1de7d87-5608-46df-85e7-f019318a3b92'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:24.979174 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3ed5a8c695456ca0edf53ad0d82bfe', 'request_id': 'req-f1de7d87-5608-46df-85e7-f019318a3b92'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:25.056321 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3ed5a8c695456ca0edf53ad0d82bfe', 'request_id': 'req-f1de7d87-5608-46df-85e7-f019318a3b92'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:25.068939 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] The token cache is being invalidate because project 6b3ed5a8c695456ca0edf53ad0d82bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:48:25.069175 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:25.069983 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:48:25.071646 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1461/2893] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:48:24 2022] DELETE /identity/v3/projects/6b3ed5a8c695456ca0edf53ad0d82bfe => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:48:34.304894 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:35.065470 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:44.070698 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:44.072221 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:44.084111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293593d-e715-4663-8793-32bc3a56fd60 service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:44.135635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:44.140523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:44.140523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:44.140523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:44.140523 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:44.149747 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:44.151155 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:44.159899 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1433/2894] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:48:48.199851 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:48.201335 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:48.213926 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7d87-5608-46df-85e7-f019318a3b92 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:48.217452 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:48.218057 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:48.218336 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:48.218610 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:48.220287 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:48.289984 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:48.291323 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af93fd-d721-4fdc-a8bb-70ed47fd99c6 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:48.300759 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1462/2895] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:48:54.155673 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=26 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:48:54.473857 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:48:54.475435 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:54.581212 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147226-e745-42fb-91fe-94346e3249db service neutron] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:48:54.587596 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:48:54.588234 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:48:54.588544 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:48:54.588815 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:48:54.590737 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:48:54.608800 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:48:54.611206 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:48:54.626939 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1434/2896] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:48:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:48:58.296558 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=30 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:49:04.621635 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=34 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:49:54.303548 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:49:54.304075 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:49:54.304188 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:49:54.322841 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] MFA Rules not processed for user `ea4ccbb465094f079dcc4ca3bcac59cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:49:54.324025 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:49:54.376372 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1463/2897] 10.222.0.144 () {58 vars in 963 bytes} [Tue Jul 26 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:50:04.369949 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:50:10.398018 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:50:10.409785 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7a6a-674b-4c23-abe0-8f94049a095e service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:50:10.415555 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:50:10.416149 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:50:10.416452 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:50:10.416726 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:50:10.418451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:50:10.431476 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:50:10.432983 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:50:10.443990 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1435/2898] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:50:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:50:13.114897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:50:13.125234 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9b6-06be-4e2a-88a9-15cdc4bd3705 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:50:13.129042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:50:13.129652 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:50:13.129935 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:50:13.130211 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:50:13.131971 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:50:13.195310 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:50:13.196786 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2517b-8bbc-4ab0-97e3-6897e3613216 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:50:13.206354 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1464/2899] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:50:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:50:20.439098 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=6 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:50:23.201996 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=30 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:10.162073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:10.164675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:10.276313 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7d343-3bcf-492c-817b-6875371071e9 service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:10.282437 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:10.283438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:10.283794 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:10.284580 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:10.286798 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:10.303302 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:10.305688 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:10.321111 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1436/2900] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:51:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:51:20.312625 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=28 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:20.463121 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:20.463623 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:20.463732 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:20.483432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] MFA Rules not processed for user `c0f8e4e0592447ef8cc91842386e25bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:51:20.484672 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:20.537370 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1465/2901] 10.222.0.144 () {58 vars in 972 bytes} [Tue Jul 26 16:51:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 16:51:20.577458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:20.578881 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:20.645964 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409e36-92d0-44b7-89c7-c893591d4361 service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:20.651562 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f380d935524a2295fd9c01a8fec95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:20.652180 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:20.652570 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:20.652850 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:20.661054 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:20.663410 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:20.671348 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1437/2902] 10.222.0.144 () {60 vars in 1389 bytes} [Tue Jul 26 16:51:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:51:25.211394 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:25.213236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:25.307477 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff226af-da85-444b-b626-d4c3adb33616 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:25.312714 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:25.313345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:25.313626 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:25.313897 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:25.315947 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:25.341324 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:25.345088 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:25.362387 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1466/2903] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:51:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:51:28.523656 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:28.534943 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8184708-1bc4-4e6c-bb41-fb41d6fe8fb5 service swift] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:28.540753 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:28.543524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:28.543524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:28.543524 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:28.543831 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:28.555908 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:28.557508 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:28.567594 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1438/2904] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:51:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 16:51:28.662146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:28.662146 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:28.743923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:28.758105 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118143-ae9c-42d4-aae2-e4bf9d4e33eb service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:28.762690 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:28.763326 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:28.763607 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:28.763880 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] PATH_INFO: `/v3/users/e57b5c97a19a42c28156f1fc82384fc1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:28.765069 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57b5c97a19a42c28156f1fc82384fc1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:28.767001 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:28.927913 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57b5c97a19a42c28156f1fc82384fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:28.928595 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57b5c97a19a42c28156f1fc82384fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:28.935265 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57b5c97a19a42c28156f1fc82384fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:28.935888 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57b5c97a19a42c28156f1fc82384fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:28.947236 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57b5c97a19a42c28156f1fc82384fc1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.017540 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1467/2905] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:51:28 2022] DELETE /identity/v3/users/e57b5c97a19a42c28156f1fc82384fc1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:51:29.254253 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:29.269724 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067105-5a99-4c12-b697-9adaa3ba8642 service placement] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:29.326814 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:29.327745 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:29.327890 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:29.328030 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] PATH_INFO: `/v3/projects/417fd6ab3a0c4537a10272ee14f1cadb` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:29.329238 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] RBAC: Authorizing `identity:delete_project(project_id=417fd6ab3a0c4537a10272ee14f1cadb)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:29.331217 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:29.421166 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417fd6ab3a0c4537a10272ee14f1cadb', 'request_id': 'req-1e161083-84d5-42b6-b0ed-b6a6c645d54c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.428361 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417fd6ab3a0c4537a10272ee14f1cadb', 'request_id': 'req-1e161083-84d5-42b6-b0ed-b6a6c645d54c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.434008 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417fd6ab3a0c4537a10272ee14f1cadb', 'request_id': 'req-1e161083-84d5-42b6-b0ed-b6a6c645d54c'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.503593 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] The token cache is being invalidate because project 417fd6ab3a0c4537a10272ee14f1cadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:51:29.503889 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.504548 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.506705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1439/2906] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:51:29 2022] DELETE /identity/v3/projects/417fd6ab3a0c4537a10272ee14f1cadb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:51:29.510718 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:29.512136 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:29.525158 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99f6b-f162-427d-b526-ff4b7e5f19a0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:29.582914 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:29.583544 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:29.583824 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:29.584098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] PATH_INFO: `/v3/users/d6f1b172408d419eba93becb942005bc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:29.585293 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f1b172408d419eba93becb942005bc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:29.587198 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:29.759703 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f1b172408d419eba93becb942005bc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.760534 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f1b172408d419eba93becb942005bc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.767535 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f1b172408d419eba93becb942005bc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.768191 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f1b172408d419eba93becb942005bc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.775875 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:29.780097 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f1b172408d419eba93becb942005bc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:29.898153 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1468/2907] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:51:29 2022] DELETE /identity/v3/users/d6f1b172408d419eba93becb942005bc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:51:30.180999 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:30.182458 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:30.195199 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161083-84d5-42b6-b0ed-b6a6c645d54c admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:30.199191 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:30.252716 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:30.253356 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:30.253638 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:30.253914 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] PATH_INFO: `/v3/projects/c60bfae6ddba400dba3276c5c5e7832a` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:30.255066 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60bfae6ddba400dba3276c5c5e7832a)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:30.257009 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:30.377549 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60bfae6ddba400dba3276c5c5e7832a', 'request_id': 'req-0e6d364f-3804-48f1-9b25-39a4936c8c24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:30.383675 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60bfae6ddba400dba3276c5c5e7832a', 'request_id': 'req-0e6d364f-3804-48f1-9b25-39a4936c8c24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:30.390445 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60bfae6ddba400dba3276c5c5e7832a', 'request_id': 'req-0e6d364f-3804-48f1-9b25-39a4936c8c24'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:30.475651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] The token cache is being invalidate because project c60bfae6ddba400dba3276c5c5e7832a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80369) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:51:30.475945 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:30.476709 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:51:30.478319 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1440/2908] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:51:30 2022] DELETE /identity/v3/projects/c60bfae6ddba400dba3276c5c5e7832a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:51:40.000577 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=4,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:40.481608 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:51:54.344667 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:51:54.346098 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:54.413740 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e494c-3d2e-4bb7-b14b-e3eb4fdcce26 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:51:54.472934 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:51:54.473553 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:51:54.473834 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:51:54.474110 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:51:54.476051 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:51:54.487763 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:51:54.489381 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:51:54.500040 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1469/2909] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:51:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:52:04.495661 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=58 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:52:10.002845 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:10.004412 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:10.116325 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d364f-3804-48f1-9b25-39a4936c8c24 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:10.123859 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6268372f3956497aae05aec64d41e848', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:10.124637 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:10.126822 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:10.127160 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:10.129028 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:10.216772 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:10.218387 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:10.237503 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1441/2910] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:52:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:52:20.234754 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=81 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:52:21.053237 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:21.054798 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:21.140150 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da7f35-f333-4274-acd9-f2c5107dccb0 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:21.143858 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7e03b6ec884bf2bf5039aefba4e5e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:21.144491 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:21.144772 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:21.145042 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:21.146904 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:21.209345 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:21.210768 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:21.221315 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1470/2911] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:52:28.344198 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:28.345763 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:28.359231 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696f532-83d2-45cb-892e-d70432b5c7e0 service nova] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:28.363711 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:28.364340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:28.364680 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:28.364954 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] PATH_INFO: `/v3/users/ea4ccbb465094f079dcc4ca3bcac59cb` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:28.366142 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4ccbb465094f079dcc4ca3bcac59cb)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:28.368223 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:28.613777 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4ccbb465094f079dcc4ca3bcac59cb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:28.614438 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4ccbb465094f079dcc4ca3bcac59cb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:28.686708 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4ccbb465094f079dcc4ca3bcac59cb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:28.687321 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4ccbb465094f079dcc4ca3bcac59cb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:28.694651 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4ccbb465094f079dcc4ca3bcac59cb'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:28.712567 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1442/2912] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:52:28 2022] DELETE /identity/v3/users/ea4ccbb465094f079dcc4ca3bcac59cb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:52:28.968120 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:28.979517 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63fccb-5583-48f5-9ef1-2d7ff2b4df56 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:29.037123 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:29.037693 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:29.037974 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:29.038249 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] PATH_INFO: `/v3/projects/94684df47fe14e55af2e72ab5d2ddc48` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:29.039390 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94684df47fe14e55af2e72ab5d2ddc48)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:29.041620 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:29.223515 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:52:29.235432 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94684df47fe14e55af2e72ab5d2ddc48', 'request_id': 'req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.241543 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94684df47fe14e55af2e72ab5d2ddc48', 'request_id': 'req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.303375 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94684df47fe14e55af2e72ab5d2ddc48', 'request_id': 'req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.313962 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] The token cache is being invalidate because project 94684df47fe14e55af2e72ab5d2ddc48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:52:29.314247 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.314877 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.316378 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1471/2913] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:52:28 2022] DELETE /identity/v3/projects/94684df47fe14e55af2e72ab5d2ddc48 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:52:29.320504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:29.322007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:29.335254 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8e73-1114-4432-b1c2-cbc872a1e1a5 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:29.393157 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:29.393865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:29.394071 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:29.394340 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] PATH_INFO: `/v3/users/14ba8146f9414dbe9f2a20c32832a29b` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:29.408969 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ba8146f9414dbe9f2a20c32832a29b)` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:29.411073 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:29.557268 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ba8146f9414dbe9f2a20c32832a29b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.557865 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ba8146f9414dbe9f2a20c32832a29b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.617245 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ba8146f9414dbe9f2a20c32832a29b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.617905 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ba8146f9414dbe9f2a20c32832a29b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.625265 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.notifications [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ba8146f9414dbe9f2a20c32832a29b'} {{(pid=80369) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:29.626376 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:52:29.643007 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1443/2914] 10.222.0.144 () {58 vars in 1229 bytes} [Tue Jul 26 16:52:29 2022] DELETE /identity/v3/users/14ba8146f9414dbe9f2a20c32832a29b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:52:29.871154 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:52:29.872604 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:52:29.886016 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3b449-a6e6-44f1-92d0-b793eb39c6a6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:52:29.943393 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd88063924d80461f80d0ee08a35cdcfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf7f222516294286bf361e9d63865e13', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:52:29.944027 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:52:29.944307 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:52:29.944606 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] PATH_INFO: `/v3/projects/36857008c0684ec9bc627668fc1cef7b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:52:29.955389 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36857008c0684ec9bc627668fc1cef7b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:52:29.957331 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:52:30.064602 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36857008c0684ec9bc627668fc1cef7b', 'request_id': 'req-efbc642d-c246-4081-8921-b00790e1938b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:30.071411 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36857008c0684ec9bc627668fc1cef7b', 'request_id': 'req-efbc642d-c246-4081-8921-b00790e1938b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:30.166434 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36857008c0684ec9bc627668fc1cef7b', 'request_id': 'req-efbc642d-c246-4081-8921-b00790e1938b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:30.177638 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] The token cache is being invalidate because project 36857008c0684ec9bc627668fc1cef7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 16:52:30.177923 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:30.178541 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-efbc642d-c246-4081-8921-b00790e1938b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 16:52:30.180052 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1472/2915] 10.222.0.144 () {58 vars in 1238 bytes} [Tue Jul 26 16:52:29 2022] DELETE /identity/v3/projects/36857008c0684ec9bc627668fc1cef7b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:52:39.765804 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=7,keystone:DELETE=2 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:52:40.174552 dv-cinder-851042-1-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=67,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:57.307504 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Authenticating user token {{(pid=80369) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:57.308923 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:57.380144 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133746a0-392d-4dec-8b03-119114f3e803 admin admin] Validating token access rules against request {{(pid=80369) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:57.439076 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c7f8c220a3844469072a97f7d2cb6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc753d253044415fbf36fa712bea0602', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80369) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:57.439690 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] REQUEST_METHOD: `GET` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:57.439972 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:57.440246 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80369) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:57.441997 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.fernet_utils [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80369) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:57.453451 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:57.455004 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b062-2fd9-4515-a84d-0ae246df15a8 service placement] RBAC: Authorization granted {{(pid=80369) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:57.464705 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: [pid: 80369|app: 0|req: 1444/2916] 10.222.0.144 () {60 vars in 1370 bytes} [Tue Jul 26 16:56:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 16:57:07.460517 dv-cinder-851042-1-iscsi devstack@keystone.service[80369]: DEBUG dbcounter [-] [80369] Writing DB stats keystone:SELECT=58 {{(pid=80369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}